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ABSTRACT 



This report of an Institute in Evaluation Teciinij^aes 
- in School Library/Hedia Progra»s is presented in outline fora to 
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participants. A discussion of the isportance of evaluation is 
' followed by sections* on wh^t^ information to collect, existing 
eva^luation and measurement instruments vith analysis of six specific 
instruments, other mea'sures of quantitative- program Aspects, and a 
description of Liesener*s costing methods* Other measures considered 
for evaluating qualitative areas of.\the program are: (1). best 
professional^ judgement; . (2) questionnaires; (3) interviews; (4) 
observations; (5) scales; and (6) ranking* The basic techniques of 
statistical analysis are described, including errors in observation 
making, computet-^use in analysis,/ maifual tabulation and computatipn 
of statistics, drawing conclusions, and the Purdue Self Evaluation ^ 
System for ^School Med^a Centers (PSES). , Sample evaluation forms and 
programs,^ bibliogtaphy, and lists of particii?,ants, advisory 
committee, and staff are appended. (KP) * ^ 
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"There has feeen a lack of hard data about the productivity of our schools, and their evaluation 
has thus been more in terms of w^at goes into the process of education rather than it's outcomes. 
This lack of simple accountability^hampers efforts to Veform public education at all levels. The ; 
need to develop ^d support the procedures to permit^aecoiintability in public education' is/ one 
of our most important tasks." / > ' 
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INTRODUCTION 



A -recent concern in education has been accountability. With reduced funding and riduced 
schciol populations, programs are being assessed for theij value to the total school ^system, aija any 
pmgram wWch is not rated highly is phased out. Some programs mafy i)e of great value toihe 
chool program, but the administrators ^and teachers who haye be^n ^involved i^ these programs 
lave been unable to demonstrate the value oi the program to the educatioji of students. They may 
have applied ap incorrect evaluation technique or may have bee^ii unaware of. technicjues for evaluation 
which woTuld/assess the value of the program to the school system. Many school liDmrian/media 
specialist^ may be in the group of teachers who aJre imawajre of techniques for^valu^tipp for, tradi-. 
tionally, evaluation for the library /media pro-am has been a circ^ation count or a record of number 
of reference questions answered. * ; ^ i 

For many years schqol librarians provided books for the students and teachers \>fithin their 
school buildings.* Although in the early, sixties 'more and rhore librzurians added forms of information 
other than books to their collections and many adopted a new tit^e^ media specialist, the emphasis 
remajined on the: circulation of materials to students and teachets. Today^'s librarian/media specialist 
lust provide a program . ' ^ - . . . 

/ ^ , ^ _ • • * Y 

designed to assist learners to" grow in their ability to 'find, generate, evaluate, • 
. .pid a^ly informatioji that helps them nonfunction effectively as ihdividuals 
ancl tb participate, fully in society. (Media Programs, p. 4) * ^ . 

* \ \ ' * 

Furthennore, the librarian/media specialist must be able to apply evaluation techniques to" the program 
to detetoine program quality which "is judged by its effectiveness in achieving program purposes." 

,(ibid.) . 'X ^ ^ '\ . • ' , 

Not only* the, stress oja accountability but also the more recefit evalilation requirements of PL'SS* 
380 (Title IVB and C) hiave'helped some librarian/media specialists recognize that tC;chniques for the 
evaiu^ion of programs and servicefii>*'the school library/media center have been, at best, irladequate^ ^ 
While personiiel in these libtary/media cerTtej^ may have been aware of and hjrv^used standards oj:, ^ 
guidelines developed by staffe departments 'oleducation pr national library and media associations as 
well as evaluation instruments designed^by re^onal accrediting agencies, traditionally, such per^oftsT 
have not been aware of nor havelinacfe e^ten$ivp4ise of other evaluation techniques. An Instit^tb • ^ 
in Ev^uation Techniques for School tibtary /Media Programs, sponsored by the U.S.t)fficc of . 
Education i^ji cooperation witn^the Pennsyly^ia School Librarians Association, was hel^ ii^Jnly, 1976 
I at the praduate School of Library^and Jnfotiliation Scfepces, University of Piftstur^>^ ; . *; ; \ ^ ' 



The goals of the institute were . * \ ^ ^ * - 

" ' tP apply an existing evaluation ^eSign to a specific situation 

« to adapt an ^existing evaluation deMgn to a sj)ecific situation 

^ to develop an evaluation instrument^ to test a given research problem based 
' \ upon a statistical de^gn' \« . • , 

. . * . . <* ^ ' * s, 

to translate data into machine readable format and analyze v^th a prepackaged " ^\ 

''compMter program • * ' • o 

' * ' * \ * 

to critique research designs . • , ^ 

to design a research project « ^ ^ 

School librarian/media specialists were given aa opportunity to leate evaluatSbrt techniques and to 
devise evaluation strategies aniinstruraerits fogr existipg.aifd future pro'girams an(4|ervices. Participants 

to becdme aware of , areas of school media programs ?md s^jrvices thit could be 
evaluated,bpth quantitatively, and q^nalitatively ... \ ' ' 

!^ ' i intwducelH tq siiftple research designs^and to the preparation of data for ; 

machine readable format a^ well asxomputer packages that |iandle data - ^ ' « 

<v ^ . . * * . ^ , * * ^ \ ' . 

^o be given the opportunity td^develop a research design, *' o 

Data werq Aialy^ted.via ^Vomptiter terininal^^ and a repbrt of tfte firfding^ was discussed. ^ ^ 



• . When the institute was funded, the suggestion was made that .this trmning method could serve as 
^ a model for similar sessions and that an outline df the activities^of this iWitufe and a presentation ot 
. the discussion questions coiild serve as a format for others whblneed to plan training and mservicf 

• ""sessions in evaluation tech^iqu^s. / * ^ • ^ . ' , ^ 

While the institute sessions wel^e recorded, outline form rather than complete text was chosen . 

• for this report. . It wa3 determined.that conceptualization in/>utline form wpuld'b>e more easily 
replicated than rfttemp|ing to Tollqwithe complete text. . 



The formal of thi^ report is as follows:. ^ * 



1.^ Basfc information is given in the nai^rative. 



Discussion questions flbr partidpanfe are ehclosedvip bqxe^ 



* . 3. Activities to be carried^btit by'the participailts aje enclosed in 
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Evaluation is the only rational approach to educational assessment and decision-making. JEvaJ-. 
uatic^mwill provide a means to mak? choices.' If What is plahned to be aii improvement is attempted, 
re^ilts of this "improvement" must be verified. Educational instftutions have long been eager to ' ^ 
. adopt innovation,;which are usually .untested or inadequately t^ted and yet eagerly attempted. As 
a result, we sjjould nqt be surprise^d to find that in 1976 the* batjHe cry^ is "Back to the Basi^." 

. The^yeajr^l976 is aisp in the era 8f accountability, tl^e need for objective evaluation of education 

, programs to determine cost effeQtivteQess. The design-of ESEA^Title IV has mSKdated the^ school 

libranan/meplia specialists' effective evaluation of the'school library/media program,. plans for change 
\^ and improvement which does or does not occur. New Title IV legislation requires the use of evaluation 

techniques to determine program needs before proposing the expenditure oPTitle-IV funds for 

* , . " Congress is consolidatinfffunds anjd stipulating local discretion in their use, has a^so f 

* required that evidence oTthe impact of these monies be demonstrate^ by thd local 
- education agency.. . . . The development of goals, objectives an3 erraluation strategies 

for measuring impact of Part B funds thus becomes a ne.ceSsary part.of the application 
process. ( Information and Instructions ESEA Title IVP.L. 93-380, Penhsvlvania 
Department of Education [1975?] unpublished, mimeographed, p. ^.') 

. A. Jackspn Stenner has defihed evaluation as '*the'process of clarifying a set of information needs, 
^Wd collecting, analyzing, and reporting information to allevjiate t^ose needs:" ( An Overview of Informa- 
i tiori Based Evaluation: A DesigiuProcedurg l Arlington,*Va.^ Institute for Dev.elopment of Educational 
lAu<]|ting, 1972, p. 4.)^f evaluation ts a "process*' then it must be an on^oingt continuous aqjivity. This"" 
process will require resources which implies expenditures^ Itjnay not bean e^enditure which one sees 
as a hscal item written into an accounting^'oumal in^the same way one would see the expenditure for 
purcl;iase 'of. testing materials for the.guidan<;e department :s evaluation of students, but the activities 
involved in the evaluation process will take the time of teachers, students^administratioQ, parents, as 
well as the librarian/media specialist. Time is an expense^ although np physical record may be^made of 
cQst df time. i ~ O 

■r .. V/. ■ ■ ^ 

I One should not evaluate an activity once, and findingit to be successful continue the activity 
year after year assuming it is' still successful: On the other Hdnd, if the evaluation activity is not sue- 
ce^ful, these results should not be stored away or ignored, but must be discussed in an.attempt to im- 
prbve, modify, redesign *an4 retest to see^if modifications coUld bring about success of the activity. Thus 
ev^uation can offset or'alleviate prbblems encountered when programs, both established and innovative, . 
appear through- testing to do little or nothing, to f urthereducational achievement* 



Ii^oday*s world, needs must be clarified with valid data. No longer can the librarian/media 
sp^cialisf exist >vith the assumed justification that the library/media center is a good place simply 
because it's always been a good place and that's true because it :s^s so tnThelircrature-particularly < 
library literature. Program assessment must be designed to clarify program needs. ^ , i 

The first step in evaluation is deciding what to collect. * " 

♦ t 

WHAT TO COLLECT ' , * ' ' 

An overabundance of information exists which might be collected., The researcher must consider ' 
three influences upon data collection. The first consideration is the amount of information to be 
collected. For jnstance, what can economically be collected? The opinions on several subjects of « 
every student and teacher in a 3,500 student high school might be interesting and informative, but 
analyzing that quantity of data could belh^practicalr' Also, if one qu/ries every student and teacher 
each time one wishes to analyze a change in ser\ ice, the public relationgjmage o/ the library/media 
center will quickly become negative. "Therefore, cfecisions must be made about what items are to be 
collected. Ii> other words you must decide what you need to know andjhow you plan to use it. This 
involves an analysis of what the problem is. It further involves how yoi/*^lan to test it. 

Researchers must also decide the items to belcbllected based upon the methods planned for 
analyzing the data^ Statisticians suggest that the method of analyzing be chosen before data is collected, 
for several reasons. Among tljese are 

' , " » 

r.. The method of analyzing shbuld dKctate the type of data collected, the amount of data V 
. , ' ^ collected and the-manner by which the data is collected. 

2. The method of analyzing will determine What the researcher can say about His results. 

^ . 3. The method of collecting and analyzing will determine if the results are generalizable 

to other pT>pulationSjL 



A 4ast consideration on what to collect is the person to whom you are to report. Evalua- 
tions are meaningless Unless they are communicated to the appropriate administrator. One musj 
realize that the recipient of the information may choose to interpret data to fit preconceived ideas. 
On the other Jiand^the person receiving the data may not be interested in the information collected and 
Irould have preferred another type of study. 
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EXISTING EVALUATION AND &1EASUREMENT IP^STRUMENTS 



*A nui?iber of existir^g'evaluation and mcasuremerit instruments have been published nationally 
by ussociaiionj? and in'dividuals. Workshop participants should/have an opportunity to scan what is 
avifmible bl it more important ly should bS able to \ - * 

Undcrslimd the type of data required by the instrument. v • 

Realise the amount of effort needed to collect the required data. 



Assess what the instrument does m^ure. * ' " ^ 

Asse^ what the instrument does rtot measure. 

Admi^iistcr the "so what" test, i. e.', what do you do as a result of the experience. 



Also, in evaluating evaluation, one-applies four criteria: validity, reliability, timeliness and cred'i- 
biliiy. Validity is determined if the information retrieved is what the'decision-maker needs. Reliability 
is osiablished lf the info npation can be reproduced. Timelines? is determined if^e information reaches 
all decision-makers who need it, when they need it, and credibility is judged if the information is trusted 
by"the decision-maker and by those he must serve. ; - ^ • 

Participants should Ulso understand the difference between formative and summative evaluation- 
and should be able to classify and evaluation instnifiieril of technique as one of the two categories or - 
a combination of methods. More importantly, participants should be able to understand how summative 

instruments might be adapted into a more formative evaluation process, 

< ' '* 

' • . ^^^-^ t 

Formative evaluation is the continual collection, assessment^ afid reporting of appropriate informa- 
lion to the person in charge of decision-making. This inforjH^n should assist in the establishment of 
valu<;^ oi priorities am^>g^tcrnative courses of action DURING THE tn^^E-OT^TttE-pROGRAM rather 
than after the program ^s completed. Judicious changes in programs may change a failing plan into a 
successful program. 1 ' 

* . • 

Summative evaluation is used to determine the ^^jectiveness of the pfoject or the program AFTER ' . 

it has run its full 'cycle. ISummative evalQation which \% conducted withouf regard to the total, content 
in which the media progi^am- functions has very seriouj limitations. Summative evaluation happens 
AFTER the program has been finished and, since no fprmative evaluation was done, no changes in the 
program ^ere made based upon evaluation. Such ch^ges^ighlliave contributed to the success of 
an otherwise unsuccessful attempt,. or even greater success of a program which appeared to be only 
modiu*ately successful. | 
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INSTRUMENTS ANALYZED * 



The fo^lbwing instrun\ents were introduced. Included here are some of the points made concerning 
what is measured and what is hot measured by each technique. Examples of the "so-vAW^ test should 
be given by the presentors ahd the pafticipants-des'cribing how the particular instrument has ct)ntribujLed 
to o^been of little use to a specific program, / 

r 

Standards as Measuring Instruments , . 



A discussion niay be held concerning the role of standards in evaluating programs. 
Points to 'digcuss include^ ^ ' . * 

i. who creates standards' * 

2., how are standards approved or afcpepted 

3. how are^sta^^dards enforced ' 

4. what progress is made as a result of the publication of standards. 



1. *AASL/AECT,.Media Programs District and School ** 



Whatis measured . 



1. quantitative figures for materials and 
equiiynent 

2. '^access to'' titles ratli^r than *'irir 
building'' collections 

2. New Jersey 



Whai isn 7 measured 



L condition of collection and equipment 

2. quality of collection qnd equipment 

3. relevance of collection to the curriculum 

4. currentness Of cdllection 



This isa sani'ple of a statc*s attempt to adapt the quantitative guides giOen in the 1969 
national stai)dard. * . . . *3 



Wfiat is measured ^ 
1. quantitative 



3..,Cal^ornia 



What measured 

L same as preceeding example 

4^ 



Injhis example, California j|ias drawn upon n ational stan dards,.but these staiydards have been 
adapted freely. The title is a misnomer as ynly three pages treat qualitative concerns. 

' ' . ' A. ^ 

*Sample pages from most initrument;5 which are discussed may be found in Appendix A. ^ 
•"♦Complete citation nTayJifiJjound in! the Bibliography. * * - 



What is measured 



What isn 7 measured 



1. Jargely quantitative 

2. introduces growth over tune: 
3 years ago, 2- years ago, 
last year, this year. 

3. measures collection size 

4. introduces location of collection: 
district or buildingjflevel 

5. introduces a grid scale for overall 
look at program ( / 



/. quality measures not measured in depth 

2. whether increase in budget was adequate 

3. use of space , ' 

4. inflation rate versu/iudget increases^ 
3. how^velM)udget is spent^ ' * * ' . 



4. Nebraska 



A 



The Nebraska guide is the result of a complete revision and compilation of many other instruments 
and ncwer4cchniqucs. * . • * * ^ ? > 



What is measu re d 



What isn 7 measured 



1. comparative figures fortbudget^ 
expenditures over time 

2. services, staffing and collection rated 

on a five point scale from' superior to poor. 

3. program components against a five level 
continuum of prpgressive quality- ' 
statements 

4. side-by-side cQi^iparison of quantrta^ive , 
figures (building level with state, • 
regiolial and national) 

5. attitudes and opinions of students and 
teachers. "V^ * ^ 



J. actual impact on teaching units and student 
%arning 

2. precise rattfigs or evidence to. general philosophical , 
statements (-mefHgyes. ^^^) for^ni^ny items , v 

3. the impact dfjec^fat money 'oftdudget and 
maintenance of local effort. 



5. AKCT 



The AKCT instrunaent draws heavily upon 
leatures but does draw\ipon the 1975 national 

What is measured , 



the Nebraska^tistrument using many of the same 
standardly. 



)i^hat isn 7 measured 



percent of quantitative figures met 
by building (measured against state and 
national) 



y. any one aspect of program comprehet^sively 
or in detail ' » J 
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6. Fulton 



What 



One of the most used. ^An earlier evaluation instrument 



/ha t^s measured ^ ' What isn 7 measured * 

I . bes^ professional judgement of / any judgements based upon objective data 

program •position along a contmuum 
01 d^riplive"9tatemeiTTS-from poor ^' 
to exemplary • 



N ' OTHEH MEASURES 01- QUANTITAJiVE PROGRAM ASPECTS ^ r 

Participants ma) be introduced to a Ijst of objective data types which might be collected in 
quantitative assessment. These measures include: i x.. !* * 

Time 

« 

Outputs 

Occurrances. 

Items • ' \ 

Successes/ FaihiKs* 

FacilitiilS (spat c)' 
I 

' ' ^ People 
^ M*6ney 




Examples: / lime , llovv long it takes to answef reference questiorw or cataTog a book 

^ How long it takes liom the time a teacher recomme^ids a filmstrip uiWil 

• ' * the library purchases it and makes it available for use. 



Outpui The liiuniber ol books cataloged 
^ The number of ciirds filed. 



V 



I he^tiumber ol books stolen. 



Qccurrcnccs The number of times a certain behavior or activity takes place 
» • (use ol card catalog, number of children looking for magazines, 

Njfiumber ol puppel^shows) . ' . * 

The niHnberj)f b^ooklalks given. 

The number of relerence questions answered. 

Tlic'number ol stories told to the-number of students^listening - 
(one story to 500 or leii-stories to five) 



Items Niynber of books, filmstrips, chairs . . . (Besf here to itandariize 

counting according jto guidelines give^ in'Seib^rt, Ivah N,, Handbook X . . 

, Success/Failure - Number of reference questions answered/unanswered. 

^ ' Number of rcfqu5ests for equipment which were filled/unfilled. 

t ^ Number of complimentsjicomplmiSS (how do,you count "golden 

. ' silence?") * *• * 

~ ' • * ■» * 

Facilities (space) Nunober of square feet for^ciixi^ area. 
^ K Numbjjr of square feet for production area. ^ ' 

Number of students who can be seated in the reading room. 

People - I^umber of staftmembers. - • 

Number of volun^^ers. ' 

Number of student*. < 
Number of teachers. ' * . 

A. 

Money , - Amount spent per pupil from local funds. 
/ . ' . Amount spent per pupil from federal funds. 

Amount given as gifts. ^ . ^ M ' 

^ : , Cost of materials. 

Cost of services* , ^ ^ * 
Cost projections of alternative methods of doing tasks. 

^ att^^ ««««««« :)c :(e «««««««««« :(c :(c :(c :(c :^ :(c :(c :(e :(e :|e 



* 



* 



* Partiqipants'need to be giyen some.experience in designing what,, how much,-and for what ^ 

* reaspn>4o collect quantitative ^ata. Practice may be givcb by having th^m use a matrix as « 

* follows: / ' T' ' - * 



WORK SHEET 



r: 
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CRITJERION OR ^ 

^. OBJECTIVE ^ 
• 


• 

WHAT TO 
MEASURE 


^iOW TO MEASURE: 
TIME ,OUTPUT, . . , 


' WHAT DOES IT 
MEASURE 


WHAT DOESNT 
IT MEASURE 
s 


SO WHAT . 


SUMMATIVE 

' OR • 
FORMATIVE 




















* 










* 






• 
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c. • 
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C V ■ ' 


0 * \ 




if 



I' . u 



CRITERION OR 
OBJECTIVE ' 



WHAT TO 
MEASURE 



HpW to*me;asure: 

TIME ,0UTPUT, . . . 



WHAT DOES IT, 
MlEASURE . 



WHAT QgESNT 
IT WE^URE 



SO WHAT 



V SUMMATIVE 
"OR 
FORMATIVE 



Increase usage of 
audiovisual equipment 
by 10 percent 



Pre and Post measures 
of equipment circu- ' 
lation 



Outif)Ut-numbet of 
circuUtions on daily 
basis on\:alendar or 
cards \ 



If circulation raises by 
desired amount 



Quality of usage, 
impact on teaching, 
whether the equipment 
is really used 
Perhaps it was merely 
checked 6ut. 



Not a very revealing 
measure of program. 

Rethink the objective 



Summative 



Ftave enough 16piin 
and spares to hand^ 
requests at current 
usage levels 



o 

Amount of usage and 
need for backup 
-quipment with a , ^ 
projector for replace- 
ment 



Time-*put clocks tfn 
each projector to log 
usage hours. \ . 

Output-Number of 
circulations of 16mir 



; §uccess/failure" 
How many requests 
for projectos could 
not be satisfied \^ 

Money-'^^ost of re^ 

placement machines, 
cost of projectors per 
usage 



Time that projectors 
are actually operating! 



Number of ptojectors^^ 
needed to satisfy 



requests 




Quality qf usage, ^ 
Who IS using the pro*- 
iJector (onl^ En^sh- 
teachers?)^ ' 



How many projecjtors 
arSTieededif usage' 
levels increas'^'i^^ause 
of inStervice trainihg? 



Gives concrete figures 
and cost figures to 
keep equipment usage 
at statu^qup level-r . 
«progragp projection 
^d improvement * 
lacking 



Summdtive 
Note: 

Could be formative 
if part of a continuous 
program of measure- 
ment, improvement, 
measiurement, 
improvement 



Cost of equipment 
for this servicer - 



Quantitative measures aredmportaht f^r jneasuring many asp'ects of program in order, to nlake 
sound decisions for program improvc^ment. W^e they do not reveal quality, quality will be non- 
existent if quantitative me&sures are not preserlt and meaningful: children can't read* widely without 
books at their disposal; reading slqll \yill not increase if progressively difficult bopks are not available 
cHilSWko. won't read unless they are give^n tune ^d encouragement to db so; teadhers won't use.lSmm . 
projectors if there are never enoudi to go around or if they constantly break do\yni ' ' 



Participants should discuss^ th^ »roughIy several, difficult but program- 
ilnportant problems: 



L deteimining how the moi"^ 
quantitatively^ I 



comgjl^ issues can be measured 



of 



2. questioning the IqJJiection , . 
counts, numbers of reference 
statistics are contribjuting tc 



normal statistics such as circulation, 
qjiestions, and others, unless these 
evaluation Sf specific program objectives. 



****** :|c *** *^ i^:* :(c * j|c*j^* ******** 

*' 

* Participants were giyen the assignment of designing as individuals or in ^ 
' * ' groups one or m|)re qualitative measures that would have an impact on * 
, * ^ decision-making. , ^ * \ * 

a. * \ ' * 

• * * 

* *♦♦♦♦**♦♦*%****************************************:{(************ * 



// J 

/ ■ ' 



UESENER'S TECHNIQUEI-QUANTITATIVE SECTIONS 



t 



Liesener has designed some valuable steps which reduce the total effort (time) and expenditures . 
for a wide variety pf services into actual cost figures which cart be very useful in program hiodification.^ 



Liesener's model (p. 50) can be explained and enough of his steps desci;^ed 



to make the quantitative measures that he advocates meaningful. 



i' 

% 




I 

* 
* 

* ^ 
* 
* 
* 
* 
* 

* 
* 
* 

* 
* 
* 

* 

« « i|c ^ 4f :fc ^fc'^f « i|c i|c i|c i|c i|c )fc :fc :fc He « 



Particippts were given a^ct of dummy figuresrfor reference materials and time 
spent on reference whictt can be placed in Liesener's Data Collection Guide^ and 

. practic<Ki|;ures follow. Afearticipants lcam to place their data into the costing 

. matrix. ^- / / 



Note: this aspect of Liesener's work can be a/dapted into a number 
of cmantitative.measurements which participants may need 
for^Heir own program improvement^uch as costing out time 
* spfeht on circulation tasks to see how much of the professionals 
d^y /and budget is being expended oh this taslc. Such analysis 
niay lead to a streamlining of circuptio\i procedures, the hiring 
of derical staff, the use of student/or parent volunteers in order 
/ -wfree the librarian/media specialist to Work with instructional 
priits or with individual stiidents pd teachers. The possibilities 
kre endlesst 



Credit Line: The 'sainple(s) from A Systematic Prepress tor PlanningMedia Programs, by James W. 
Liesener, are reprinted (used) by permission 51 Ae Aine^ Association; copyright 1976 

by the American Library Association. - • 



1 



Lcifcncr, 



',Jam.cijw., 



A Systematic Process for Planning Media Programs, piics^$o, ALA» 1976 
^id, pp. 115428 ^ ! ^ 
^/6id,pp. 129-151.^ 



1'^ 
f 
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• PRACTICE DATA FOI^ COSTING OUT.REFERENCE SERVICE 
. WITH U^SENERCOSTING METHODS ^ * 



REFERENCE HOLDINGS 



ENCYCLOPEDIAS 



BAS|C TOOLS 



INDEXES 




4 



HOLDINGS 


NUMBER 
ITEMS 
REPLACED 


NUMBER 
ITEMS 
ADDED 

> 


TOTAL NO. , 

ITEMS 
PURCHASED 


COST 
RANGE 


AVERAGE 
UNIT 
COST 


4 sets 


1 set 


none 


t 

one 


1 

K 


$225.00 

* 


175 


10 


15 


25 


$5.00-25.00 

'f 


$13.00 5 

• 


h 

(reader's 
guide) 


1 


0 


• 

1 ' 


$25.00 ^ 

1 


$25,00 



EXAMPLE: 



Hour 



Activity 



Day_ 



> 9:00-9:45 > 
^9:45-10:00 ^ 
10:00-10:30 



J>ate^ 



Hour 




Activity 







Hour 



Activity 



JDate^ 



^Hour 



Activity v. 



• 20 



u 

D 

n 

u 



.B. 
C. 
D. 
E. 



U.F. 



Categories of Reference Service Tasks ' • pesener ' 

' • • ' . J 

I * 

Assistance in identifying and locating inaterials in centers 
* Assistance id identifying and locatbg m&terials not in centers 
Alerting th|'user and current awsUcgness services / 
Bil:>liographic and searching assistance * • * . | / 

1. computer searching fees or time- costs ■ ' - .* 

2. assisting users in compiling bibliographies , * j ^ ^ 

3. performing simple subject searches , i • ^ * ^ 
4^ compiling exhaustive bibliographies ; . ' . * 

5. evaluating materials ancf/bffprepafing annotations or critiques^ 

6. preparing bibliographies or state-of-the-art review * " 

7. " computer searching activities * I * - 
Answer service^ ' /I 

L answering simple fact questiqpf$ in person or by Imail*. 

2. answering more complex an<i time-conSuming q^iestions ^ 

3. using informational retried systems for answenli^ueAic 

4. conducting research* and development^studits ^ ' ,j 



r " 



ions 



Sample Data Collection 



CATEGGrkf 



TIME 



Professional 



SERVICE OUTPUT 



OPERATIONAL UNIfS, 



II.B. 


38 min. 




19 


II.C.-! 


'lO.min. 




2 


ILD. 1 


■ ,1 hoAr 




1 


'iLE:2' ' 


10 min. 




3 


ILE.3, ' 


15 min. 


i- 


4, 


n.E,5 


.35"iHiir 


■Ik 


^ 1 


II.F.1 


40 min. 




15 


ir.F.2 1 


20 min. 


\ 


3 










II.B. - 


20 min: ' 




16 


n.E.3 


30' min. 




,8' 


II.E.4 


15 min. 




' 1 


II.F.l 


1 hour 




30 


•II.F.2 


35 min. 




3:. 



Day 1 



D^y 2 



UNFILLED REQUESTS 



4 
1 

5 

.6 
1 



CATEGORY TIME 



SERVICE Ol^UT 



OPERATIONAL UNITS 



n.B. 
n.F.i 



U.B. 

ri.F.i 



II:B. - 
U.F.1 



UNFILLED REQUESTS 
t 



n.B. , 


10 min. 


. ,7 


n.to. 


•■ 20 min. 


1 


n.E.4- 


•30 min. 


1 


n.E.5 


. . 45 min. 


2 


n.F.i 


ae-tnin.. 


16 


n.F,2 , 


20 min. - 


2 



Clerk 



^1 ■ 



Day. 1 



L'hour 45 min. 
30 min. 



1 hour 454^n^; 
45 min. 

/ 



1 hour 45 min. _65 / 
45 min. 18 




D ay 2; 



Day 3 




rPh)fessional. works 7 hrs/day.ySalary/^14,000/year 
' ^ ' school year =180 4ays 

1 Clerk works 7 hrs/day. '^alai|^ $5,000^^ . * ' , 

. , . school lyear =T 180 days , \ ; ' ^ 

' • ■ . ' ' * . ^. i 

Total budget for materials, equipment sfii)plies^Tcpair, rental = $5,600.00 / 

Note: Other figures are needed for final program costs. " ^ 

.TOTAL PROGRAM (SERVICE) (CO|TS > ° 



10 
4 



16 
8 



14 
10 



'I 



I. -.ACCESS TO MAtERIALS,,EQUIPKiPNT, 

AND SPACE 

II. REiPElffeNCE SERVICES 

III. PRODUCTION -SfiR VICES ^ 

IV. instrOction. 

V. consulting SERVICES , ^ 



• STAF^.OliLY 


TOTAL 


PERCENT OF 
TOTAL COST 


-VALUE . 
IN PERCENT 


1 

,$5,986 


$.10^86 • 


41.Q r^<« . 




. 7,749 - 


8,324' ' ' 


' ^ 




y ' 2,500 


4,000 


16.2 1 






^ i,140 






570«» 


57Q 


5.0 





OTHER MEASORES LEADING TO MORE QUALITATIVE AREAS OF PROGRAM j 



.Best Professional Judgment 



Many evaluation. techniques incfeide best professional judgment as admeasurement tool. It is 
mo^t often used wth reg^onal_ accreditation teams who examine quantitative and descriptive data 
collected by a school, make observ^ations of their own, and tlien use best professional judgment to 
draw their evaluative contrlusions. Obviously, the person d<5ing*the^evaluation is important. 



Best professioifai judgment is done by a judge(s) who 

a. may he a friend-or an enemy 

b. has much sympathy or Ijittle sympathy for the m^dia center program 

V . 

c. has an ax to grind 



m^d 

> 



d; wishes to be very supportive, of improving program 
e. compares your progranji with his/her ovm 

^f. 'will be rated by "best^ professional judgment" (perhaps by someone 
in your school which is noW being rated) in the near future. 



Participants will want to review the sttengths and v^eakhesses of outside evaluators, 
i.e^y-what*s good libout an outsider cvaluator, what jcan and cannot be expected 
of them, what to, do and not to do^vhen you are an eviduator. 



What are the advantages of an outside evaluated? 

1. An outsider often carries more weight*wth the decision makers since 
administrators may have more confidence in the outside^^evaluator. 

- 2. An outside ev^uator should have a'less biased^inion^and should be 
less Jikely to heed to make things look good/ 

' 3,^ The outsider may not be confiJied to the limits which could be imposed 
^ — ^upon-4h€^n^d;use s^ff membei^. The outsider could- be able to evaluate " 
from a wider perspective; ,1.%' . . * 

) 4. The administration may be more likely tjrconsider a report written 
by«an otrtsider who seems to carry mp¥e prestige or authority^' 




What tan or cannot bfe expected of the outside evaluator? 

The outside evaluator cannot possibl)^ see the entire picture^becatlse. the visit is brief. 
Real issues could be missed. If the evaluator is a practidng professional he/she could make ' 
an unconscious^ c<^parison to his own center, other places he has seen, orlhow he thinks the 
program should be managed. \ ^ 

The outside evaluator cannot FORCE the admth^stration vis-a-vis recommendations as^part 
of the fmal report to hire staff, increase budget, and other "miracles** which librarian/media 
specialists would enjoy ^ven thou^ such.a re^icommendation is based upon an analy sis ofj tHe 
present situation, * , ^ . 

What should you do and not do wlftn you are asked to be an outside evaluator? 

^ If you are asked to act as an outside evaluator jwhat do you look for in making your 
^praisal? ^ - ' 

the look of .the facility; the atmosphere of the library/media center. 
* i* ' ' . , 

quality of collection?* Are titles leftovers from Arrow Book^CIub, parent 
donations' from the attic or basement, discards when the high school moved 
to a new building, or free filmstrips from pudding manufacturers? 

.students- seem to know what^to do in the^l/rary/media center? ^ 

students and teachemseem happy to be in the library/media center? 

^bulletin boards are attractive in the library/media center. , . / 



library/media center is reflected in classrooms in sights and s^ounds of library ' 
materials in use and on classroom bulletin boards. ' " ^ ' 

evaluator asks pre-determined questions of stuilents, teachers, staff and ^ 
administrators.* • » \ 

apply evaluation forms designed for the visit or other evaluation forms. 

A variety of evaluation techniques are available to the researcher. Amrag these are interviews, 
obs^ations, scales, and ranking. Explanations of these techniques follow. * ' ^ 
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Questionnaires 

Questionnaires are a group of printed questions used to' elicit information from respondent 
by means of self-report. Questionnaires have s^veraj advantages over interviews: 

less expensive . ^ ^ 

* * 

^ * fcasy of preparation and distribution and tabulation 
^can be distributed \yithout interviews , * 

9ffers respondents an opportuhity to prepare and revise their answers. X 

Researchers in school library/media centers are unlikely to use the most popul^uf-the fnail 
questionnaire-unless you mterpret puttmg a questionnaire mto a teacher*s box as "mail.** 

A "mail** questionnaire is susceptible to*: ^ / - ^ 

gross mis-interpretatipn ^ ^ * . * 

self-sdpction of respondents . * 
carelessness " ^, * \ ^ 

^ , other forms of abuse " ^ ' * 

The researcher should confirm all data received and be suspici6us of all data received; • 
% " . - • "^"^ 

/ , Some other problems occur with questionnaires: 

J ^ 1. seeking. data which cdijld be better determined other ways; 
i V asking opinions rather than evidencf ; asking for great quantities of ' ^ 



information.. ^ < X 

2. . respondent usually doesn*t knbw invj^tigator^ what he means, what he • 
wants to kngw. You may be linaife ori.mistaken in judging ^om the . # 
^^^Mrespbnses wTiat is meant by th^ answer. " , 

3. how to* get honest answers , >, 

a. ask questions in two different ways ' . \ ' 

. guarantee anonymity which may^ bring more candor or less 

* concern for truth' ' ' T . ' ^ * 
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If questionn^re is your BEST means \ * " *" 

1. you "must make questions (and answers) clear to your respondents (and 
yourself) and as free as possible from different interpretations 

2. maximize the number of returns. - 
lo msure clarity m questions: 

IS.. ^ . ^« 

^ 1. provide careful directions to the respondent as to how tOv complete the 
- ' form and report the answers. • ' - o ^ \V 

-.2. define^all technical terms. 



3. any deviation fron^ ab0^te%jihology should be reported to the 
respondents ex: :jn.^dta^^ertter rather tlj^ 



4. offer respondents a se,t of possible answers, but this stiffles creativity qf 
answers ifi many^ respondents. 

5. ^oh the other hXdralloVi^art^^fJi^l^'or ftee-style ^Iwers may reguh in 

non-comparable answers. * ^ -.^ 

, ■ ' ' ^ — \ ' ^ 

6. read questions aloud to elementary students. ■ v . "\ \ 

Pretest questionnaires to determine if respondents ur^dfei^od wh^t wa^jjaeant by the cj^Uesti 
and if you interpreted the answers correctly. If you change the qliertions, you .must again 
pretest. .Hie pretest will help you 

^ rework. , 



^- ■ ■ - ■ y ■ ■ . 



^ 2. secure information about length of time it takes to complete. 

3. help compile aj^wers to open-end qyestions which could form the ba^is 
^ ' for a check-list of answers. , 

To maximize^e number of responses, motivate those who receive the form \vith the promii 
of a compilation of explanation "^Df the results. ^ ^ . 

\ 

intezyiews ' * t 

The interview is the purp6seful conversation between the researcher and the respondent 

Advantages > . • ^ . 

1. You get the. information directly from the person rather than vicariously through a 
• paper and pencil exercise. ^ ' , ' ' 

2. You are able to see that the person understands exactly what yoii need to know 



isadvaritages ^ 

♦ 

1. It is, time consuming. » * ^- 

2. 'Data can be very difficult to code and reduced to qyanjitative measures. 

3. The interviewer can ruin the data by introducing subtle or blatant bias. 

Types Of Interviews . . - ^ * 

Structured (little deviation form planned questions) • " ^ 

a* Fixed ajtemative type: (Do you like' to re^? a* no, b.- sometupes, c. yes) 
~ 1^ (How many books do you read a week? a. 0, b. 1-2, c. 3-5, d. over 5) 

b. *Open ende^ items using a funnel question which guides or gives a situation: 
(Compared to other scj^Uibraries^yQu fiave nseH, v^hat ar^ spm^ ^^fivifiP!^ that.. 



we provide that youTi^rthe most helpful?) 

c. Scale items - the^ questions present a'statement and seek some form of agreement 
(I find it easy .to check out a filihstrip projector to take home. SA 'A N it> SD D] 

Whom Do You Interview '/ ^ * 

•1. those directly involved in the project 

2. those who should have been involved but were not for some reason 

How Many Interviews Should You Schedule 
^ 1. All those involved if the group. is small 

2. A sartiple of tKose who were involved or should have been invofved 

Do*s And Donets For Interviewing 

il . % ' • " • 

L From Kerlinger, pp..4o5-86 * « , 

' ' , ^ 

a. Each question mVist lead to the research ^objective (not counting ?h.ose qifestions 
designed to 'put the, person at ease, etc.) . ^ ' • 

b. Ask the best type of question for the^inforrifeition ybu lt5ed^(ppen ended, scale, 
or fixed alternative) * * • * 



c. Make die questic^ns clear and unambiguous (one idea per^q^estion) 

- bad: What do youJind the most/un and the most useful when you come to the m'edi 
^ center? ^ - f' ♦ 

better: - M^t is thfmost fun about coming to the imedia center? 

d. Don't ask leading questions ^ ^ ^ ^ ^ * 
bad: What isithi^mtMtfuif ^out coning to the media center? 

bettcfn Do^you have fun when you cqnie to the media center? if yes, what things 
. do you do in the media center thataffe the most fim? ^ " 



e. Don*t ask things the person would not know • • 

bad: Is the media specialfst well qualified to hold this position? Asked of a ^ 
studentior-teacherr' . . ^ , 

V Be careful about sensitive areas - use soft wprds rather than hlursh ones 
'bad:_Why do you think Mrs. Jones yells and screairnsiat students?^ 
better: Why do you think Mrs: Jones shows tlfiiVI^^ is^upset with students 

gADon't make one answer socially desirable ■ > 
''^)adf Mrs. Jones really knows how to tell stories well< Do you enjoy them? 

2. FromG&ldhor . • 

-■^ , ' • 

• a. Don't put too much* distance between yourself and the person being interviewed 
^ (dress, manner of speech, apprqach, friendliness - lack of) 

b. Don't reveal your own bias 

bad: youranswer hit the nail on the head , • . 

s 

c. Don't interpret the answer you hear in terms of your own bias 
- • " (what I think you said is definitely wHat IVe always believed) 

bad: He said, "J hate libraries." t heard, "libraries", and replied, 

"they are lovely places aren't they?" ' - - . _ ' _ - 

^ d. Try to check on the honesty of the person (follow-up questionnaires, observations, etc 



How Tq Code jResponses •* • 

1. Fixed alternative - ea5y - h^ve coding instrument to mark during the intervie\^ 
Do you like to read? 



boy 



girl- ^ 



yes c Ao . 



2. Take notes - then try to Cafegorize the responses intd" patterns 

3. Tape record - transcribe - then categorize' (extremely time consuming but moreVccurate) 



-Ethics 



1. Keep the ^responses annonomous if that is apprqpriateii 

2. Have participation voluntary 



i 
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Observation 



/ 



Observation as a technique in research is the collection of data by the researcher who~ observes 
the phenomenon, first hand. , ' 

Techliiques 

1., Decide on the categories of the things to be ojDsecved. You should be selective. E^ch 
. category should be defined carefully and some instances of it listed for the observer so 
that he knows how to categorize it properly. 



2. One need not observe all day every day. Take samples t>f time periods until you dre 
H confident of representativeness of the data. \ 



3. The observer may rate behaviors or occurrences on a scale' 



unfriendly friendly 
1 2 3 4 5 



efficient 
1 2 



' inefficient 
.3 "^4 '5 



caution: rating scales ^an produce much ,error: 

1. halo ^fect (I Hke youXvou can do no wrong) 

2. severity ( I don't like you^^^u can do no right) 

3. leniency (Hef'citation from tht-Reader's Guide vfas almost correct - I'll give her ct^'dit 

4. central tendency - (she's about an average story teller) 



Do's And Don'ts 

Trim the bias of the observer down to a minimum 

* ^ ■< 

2. The observer must be knowledgeable enough to bd able to recognize the behavior*' 
or the occiir^ence when he sees it. _ / 

^ 3. D6ri'^^^vor^y too much about the observed acting too differently when they are being 
. observed. 

4. Don't try to have the observer try to cojiect so many categories of behaviors*" that the 
error rate be.comes unnacceptable ^ / ^ - ' 

-I . 



5: Be sure that you ate measuring what you think you are (George is reading his textbook - 
actually George is reading 4 comic book which is inside his textbook) - ' 

What Kind Of Data Might You Have 

i. quantitative , ^ , • ' 

2; ratings (best professional judgement) , ' • • ^ ' 

* 3* ranking ^ - ' ^ 



OCk 
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"Scales 



Fxequency 

' — YET 

Regularly 
Daily 
• Always 



Sometime yes ^ 
Occa^iorially 
Weekly ' 
Frequently ^ - ^ 
Not operational Partially operational 
Very extensively Considerable 
Pbserved Not observed. ^ 



Sometime no 
Rarely * 
Monthly 
Occasionally 
Fully operational 



Some 



No ^ 
Never 
Yearly 



.Seldom 



Very little 



Never , 



Not at all 



Value Or Approval 

Superior Above average 

Very important Somewhat important 
Entuely satisfactory 
Mucli improvement needed 

1 L_ 



0 0 e-^. 

Average , Below average Poor-missing -^'^ 

Of little importance Of no importance 

Usually satisfactory Unsatisfactory/needs improvement 



Little improvement needed 



Degree of acceptance! . * 

Unacceptable Questionable Accept witli reservations Acccfjt in general Endorse 

completely 



Degree of implementsrtiqn: 

Not Weakly^ , 

implemented impleinuSnferd: - ^ 



Adequate 
Excellent < 
5uper cool 
Like 



' Inadequate 
Gcyod ^ 
Cool 



^ 'Average ^ Strongly ' Fully 

implententation ^implemented implemented 



Average Fair Poor 
So-so Gross Super gross 
: : Dislike (the semantic differential) 



ERIC 
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Attitudc^cales ' , * 

* * * 

^ Note: One must realize respondents may not always be honest. . 

. A. Lil^ert scales (ordinal scales)-jV 



These arensecf to measuire attitudes,^d iisually employ five choices expressing agreement. 
You may use other choices than the words **agree" or **approve- ' - * - 
Example: 1. SA A UN t) SC| • ^ ^ < . 

2. 1. elementary 2^nior high 3. senior high 4. college graduate school 

B. Thurston scales ' . ♦ * 

These are used to measure^interval data. They may also measure favorableness. 



They employ equal appSaringlntervals and ivoh^iidgeswho Vate each item. An example 
* would'be student scores compared to judge^mSSian score or/*scale value.". These always 
^ use many itcfms. ' 

Example: • ^ ' 

Give a rating to each stateihent: 
' I believe th^|^edia centers are essential for effective teaching. 

disapprove neutral % approve 

. 1^ 2^ 3 ,4-^ 5 6' 7 8 9 lo' 11 ' 12"' 

I go to the media center because it is quiet. 

disapprove neutral approve 

- ^1 2 3 4 5 6 7 8. ' i/lO 11 12 * 

I think that too much money iS' spent on media center materials. f - 

disapprove — neutral approve 

1-2 8 4 5 6 7 ' 8 ' 9 10 11 . 12 

C. Other items to consider: " ' 

. ^ ' 

A. Should your scdejndUde the "don't know'* or "no opinion" tatings? 
• Advantages: provides rater with a *Svay out" 

* pro<ddesr less errof with otherYatings 

Disadvantages: more difficult to analyze 

lets rater hedge^ . — 



B, Positive to negative vs. negative to positive scales-which one3 and why? 

a. yes -= — -no; g^Q^ — --bad 

b*. no yes; • * baC--^ ^-^good 

Statisticians usually recommend positive to negative. . , 
Sometimes negative to positive is good. For exaipple children answering 

V 

Do you like to go to thelibrary • * ^ jP^ ^ ^ < 

a. yes sometimes no ' 

h.\ n<f sometimes yes . . , 

scale is better choice since most children will mark "yes" to scale "a" withjgi^t 
really considering the "s9metimes" and "no" responses. ^ * ;v 

' ' ^ Ranking , ' 

* 

With this technique a person is given a number of items or statements and he ranks them 
according to , 

' agreement ' ^ ' 

value 
frequency 



For example: As a classroom ifeacher knowing that you can have £inly one specialist in your building 
next year, rank' the followiiigsp"e€ialists from 1 to^4 (1 is higheA)^ccording^to which you would 
rather have the.adminis^tration employ . ^ 



physic^ education teacher 
music teacher 



/ 



art teacher-^ ^ 
j^>""' media specialist 
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STAxisTICAL ANALYSIS 



Statistics provide systematic, objective procedures for collecting, organizing, sum: narizin^, and ^ 
analyzmg large quar^tities of data that can be quantified. In addition to this, statistics reducerthe 

^ sheer volume of d^ta to summary values which can be displayed on graphs or charts, or utiliaied m 
decision making to greatly enhance the objectivity of that' function. As such, statistics faqiUjaltc; com- 
munication and interpretation of what th^ata ai'e really saying. There are even statistical prpcedures 

' ^ whichpermit the extrapolation of findings tkr beyond the restricted setting in which the r^earch was^ 
actually done. 



The students have spent part of two days discussing the appropriate data to^bjc^ collected 
for pyogram evaluation and a number of instruments were introduced for measuring 
and qollectin^ data. / ^ ^. 



• - ■ . .■*■;■/ 

0 . The actuy data reduction and analysis was done oh computing facilities at the ^University of 
PitfsJ^urgh. The Statistical Package for the Sdcial Sciences {Sl*,SS)i was the particular.package of statistical 
programs utilized by the participants. • ' , ^ , ' 



The objectives of this section idll-be to gain some understanding of some of.thp basic techniques tliat 
are applied^in statistical analysis, whi^contentrating on two of the simplest and m^s^ useful types. One 
is the chi-si^uare test of independence^ the other a t-test of significant differences in the measures on two 
groups, oy the s^e group if two measures, one pre and one post, are to be^used<^^he I^^r is comihonly 
used in evaluating experimental programs whereby one is interested in whether or not a new material, or. 
a new method of instruction, or a new curriculum, has beeo instrumental in^imtiroving the perfomiance 
of students in the experimental program. The question being evaluated js, "Di^d the students in the 
experimental grpup perform better (score higher on a standardized test) than students who received regular 
instruction?*^ The difference between a student's pre-instmction performance ^d his post-instruction j ^ 
score ifs called a gain score. The evaluation of the experimental program is based on differences in gain 
scores between the "experimental** group who had the new inaterial o^ fiKi^thod or instruction, and the 
"control** group who did not.^ But before this wil really make sense, dnct^eeds to know a little something 
aboyt some'other concepts that interrelate to give the data analysis validity, reliability, and creilibility. 
>*ng these are: • <l« ^ w / • • 

Some me^urement theory • , ■' 

The concept of a distribution . * '- ■ 



Norman H. Nie, C. Hall, eu al, Siatistical Package for the Social Sciences, 2nd ed., McGiaw-Hill, 1970. 
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Errors jnlhaking observations 
N^easurement scales 
Sampling theory , 
Statisticjd hypothesis testing 
Research design 



The physical scientisP 



X Observes a phenomenon ' 

Decides on the properties to study ' ^ v . 

]Eormi4ates hypotheses about the properties relevant to the problem and > 
ignores others ^ ^ ' . 

Collects datVand employs the scientific method to see whether or not the 
data suppprt the hy]()othe^s 

This is fine for studying phenortiena/objects/organis^s with fixed propeipies which can be measured 
several timajto check the accuracy of the mfeasuxes. • \ * . • , 

Other phenomena, especially*those in the behavioral sciences,cannot be studied more than once 
in the same organism or object. The process of observing or measuring the phenomenon changes or ' 
destroys the very thing that is being measured or evaluated. Controls must be introduced to insure.that 
the only thing which causeda;significant increase in the gain scores for, the experimental group was the 
new ^tenadsjc>Mt[CtHo^^ ' - . - _ ^ ' _ 

It is especially difficult to measure and assejss human respt)nses to stimuli bccause^iJimans never 
reactvto the same stimuli exactly the way on the second encounter as they did at the first-a little 
faster^ a little slower, but never exactly the same. 



Longitudinal studies, or time-series studies, which measure change over time offer a great deal 



of potential for evaluation assessments. 

■ V ^ : 

Sources of Data for Evaluation Assessmeiits 



Effort is made to make /observations and data collection under as natural conditions as 
possible* ^ \ C • ^ 
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-: Ratmgs— — " ^ 

Surveys - . " ^^S*. ♦ 

^ ^ Inventories or pencil ic paper tests 

- Recorded data -"psychological functions * , 

. Errors in Making Observations V • , 

■ ■ ■ " \ * ■ 

There are a niitriber of ways that sources of invalidity can work their v«ay into the collection of d 
Some of these are errors of observation. Some are*humai^ errors in Whicb«tiae tnind fools the observer. 
Examples of this psychological phenomenon are: . A ' - 



Habituation 



\ Expectation * ^ 



Suggestion ^ i , * * / / ^ 

Even when great ckre Iras been taken to insure the accuracy of the measure^entsj and the appropriate 
controls have been applied to the experiment, there still may be disagreements between measurements 
which are repeated under identical conditions. These differences arc^pnsiditred to be randoip errors 
of measurement and are attributed to chance. 



. Me asuremertt Theory . * - * 

Define: Variables and Constants - ' * 

Differentiate between: Discrete and continuous measuref 
"Measurement Scales, . , 

Nominal*;/^- ^ ^ 

Ordinal - 

Interval * • ' • . 

Ratio 



V 



Interp«ing Data With Descriptive Statistics ' 



Descriptive statistics are quite easy. to understand and very valuable in examining popi^ation 
data. Worlcshop participants should understand 'the^ meaning of popul^ttion (measuring all persons 



•V . 30 " 



' J-y 



within the target group). Valuable techniques for looking at data indude: ^\ ' 

*L frequencies (23 said **.yes", l^^said *W0 

^ , -2. graphingidata (bar graphs, line graphis) . - v ^ ^ 

3. percentages ' ' 

4. ranking ^ , . _ - ' 

5. measures of central tendancy ^ean, median, mode) 

6. Indicators of^r^^ative positij?nin a distribution (PercenQle, standard deviation) 



Many workshop participant^ already are aware of and can c^ntpute most of the measures 
listed abo^er What needs to be emphasized is that descriptive statistics should not be sold 
^hort because they are^^ple to compute and are meaningful for interijfetatibn. If 
participants are not familiar with the above measures, some simple examples can be used 
fronr^ actual school library sitSatioiis *to iUustrate the us^ of the statisticsr^-Eoi^example, 
compute the. percentage of childrcfn Who read at least 100 Books during the school year; 
graph usage data* of 16inm projectors to analyze peak usage loads and thus predict the 
needed numbers of projectors at various times^of the year; have children rank V^hich of 
ten bqdks they like the best' and Compute the winner ftpm* their markings; what is the 
per(?entile of children computed for libraryj^iUs on^ii^johal te^s? 



Interpreting Sample Data with Statistics ^ \ ^ ' • ♦ -'V^^; Z^: ^jc^* 



\ 



Sampling Theory — t . ^ •^''^ *> ' 

When all members of. a population, cannot be measured or whdnMt.wgm a1>e too e xpensive or 
impractical io do so, a sample of the group may h% mtasured^to represent the entire populati^. 
Almost aJl sta'tistical techniques are based on random samples of data,/ AH researdiers need to know 
something about sampling and S2(rnpling theory ami how random Samples are obt'airxed^ Familiarity 
with computerized random number generators or t^l^ of random. numbers is esseh^al. ^^f- 

Strategy for Making an Objective Decision Using a Statistical Analysis I * ^ 

^ A test statistic is decided upon. A sample of data ate^ collected and the statlfetic^ is computed 
and compared with the tabled value forjthe appropriate degrees pf freedom to see if the value of 
the test statistic falls in the region 6f rejection for the null hypothefei^. • • ^ 



Workshop participants should 'discuss the problems that will 4iormalIy arise in using 
sampling techniques in a schT)ol: . ' 

Problems of sampling within a class: ** ^ 

lT-5ffl»ple^ze will usually be too small unless the class is very large (100 or more) » 

2. ,it is yfery difficult to have an experimental and a control group Within a qiass 
because they , will jnteract with each otfier in ways that can Influence the outcome. 

Solution: sample classrooms >. 

1. determine tne number of sections"(heterogeneous groups) 

2, are cljisSes biased^on treatment variable before you begin? If yes, thraw . 
biased classes oiit frpm research consideration. 

randomly assign classrooms to treatment ' k 

4, if a questionnaire is to Ibe given, one third.tjo one haif of the classrooms should 
be sufficient ^ • — ^ — 

5. if classes are hbniogeneously grouped then you must have indivittOHls from all 
levels. Many times this will require that a few student^ must be selected from 
each group. If there happened to be tw^ classes on each ability level, then one 
could be selected from each level. ^ I 



Warning: Befpre proceeding with .an , experiment, questionnaire, treatment,' etc., the researcher 
mu'st have cortfidence that this sample is representative of the populatioii and is not biased in a.way 
that wiiy^ew the >outcome. There We sophisticated ways to check repres^tativeness but they may 
be beyoiW the scope of a short workshop. In the absense of these techniques, a commoo sense judg- 
ment will have to suffice. ' » < * , ^ ^ 

The T-Tcst . ' ' 

The T-lcst is ^ very useful and simple test to understand. Sample data are collected so that a 
me^ can be computed. For example, two groups of stucj^ts are given two different types of Hbrary 
skills instruction. Their test ^cores can be added up and Iraie^n score for each group can be computed. 
The question is whether the difference in the tw^c^^anibpre^'^is stati^ically significant. The T-test is 
computed and Compared with the tabfed value CefHthe-approdnate degrees of freedom to see if the . < 
valCm of the test statistic falls in the region of rejecnon^or the null hypothesis.. . _ * 



^ ■ *( . ' • ^ • . 

If the value of the test statistic is 'not large enough (larger than 1.64 jn this case) to place it lA the 
region of rejection, then there is not enough evidenicq in the sample data to conclude that the material 
or method of teaching resulted in a^diflerence in gain scores lar^e enough to be statistically sig^ficant. 

•I. 

In the example above where there are two different methdds of teaching library skills used, one 
may determine by the t-test that one method is better than another. One must compute not only 
^ statistical significance but also practical significance. Perhaps one method involves computer-assisUrd 
^ instmction at a co%i of $5,000 per student for one year..* Perhaps^he other niethod utilizes paper and 
pencil exercises coupled with filmstrip's ^d audiotapes and costs*$10 per«tuderft per yean A trade- 
off judgment must be made and thafjudgment is'npt always a simple matter, , 



The Test of Independence 



This statistical test detemjines whether or not theMistribation of responses on one variable is 
independent of the way the individuals responded or were distributed on a second variable. * 



^- The data are arranged into a-bivariale frequency distribution. \ 



Reading Ability 



X2 = 6.52, df = 2, p<^.05, ^ = .32 





0 








Sex 


Good 


Average 


^ Poor 


■ ^ Total 


Male 


10 


24 A 


16 


50 


* *• 

Female 


15 


23 


. ^ • 12 


50 




. 25- 


'47 


■ ■ 28 
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- 33 



^ This information will bp"^found on ^Ke computer printout after the participants run their pro-ams* 
It helps the researcher to determine objectively whether or not^the way the students were distributed 
ovefreading ability was independent of their se;c. If the nuU hypothesis of independence is true/X^ 
\with 2 degrees of freedom (df) will look lilce this: • ^ ^ ^ ' 




R<!gion of 
Rcjcction(^ 



5.99 



The probability of getting a yalu^ that lai^ge or larger when thejiuU hypothesis of independeacf^' 
is true has a probability less than .05. Thus it would be concluded that reading ability is not independent 
of sex for this sample of data. \/ . * ' 



' The test always requires freqiJtencV data and thaUrequency data must fitWo a table such 
as the following: / ^ ' » *^ 

■ ( _ • 

Booktalks were given to two groups of students; one in person, the otfier via 
television. Students recorded the jiumber of books they read two weeks b^efore 
the experiment and for two weel^s afte/the booktalks* (The \^eks were care- ' 
fully chosen to avoid other stimulation or deterrents to reading.) The number 
of books read by each group after the month *s experiment are recorded in the^^ 
following table: 

Booktalks 









\ 

In Person 


TV- 


Number of 
books read 




Before " 


35 

V 


40 • 






• 

After. 

/ 

J 


; - 65 


42. - 



'x2 = 7.063 



df = 1 cnti(2l value'~3.8 



Therefore, since our valu^ is greater than 3.8 it is si^ificant. inspecting the cells w< 
that the in-person method of booktalks pro'fluced the .best results/ Pfaaical significance mus 
be judged. Did all students in the "in person** group read more or did just a few? What perc 
the students read more? What was the mean increase per child? (This could be tested statist 
by me and of theH-tesi.) , the time used in giving in-person booktalks worth the results? 

^ ' The Computer as a Tool to Aid in Analysis 

*********** ♦♦♦♦♦♦^e*********** ********* 

* A handout containing a model computer program from the Statistical Package 

* for tht Social Sciences (SPSS) was distributed to the participants. 
************************************** **********j|^******************'* 

The cc^cept of "canned** statistical programs stbred\n a library was presented. Job c6 
cards to call out and use these pre ^r^ns Were explained. Organizing the data for keypunchii 
the da.a processing cards and the appropriate fonnat statement to n^ake tlieixfafds machine i 
were examined in turn. A verbal introduction to the available computing facilities, ihcludin; 
card reader and printer in the Remote Job^Entry (RJE) Station in the building where the ini 
was being held, followed. ' « - ' " 

**************^**************'»S******^**************V^ 

* Participants separated into groups l;>y level, interest, and developed a projeet 
• * based-upon needs. Sample data were generated and applied toNa worksl^et. 

* ' Participants then key punched data cards and program cards, ^ta were ; * 
: ^ *' aiialyzedjhrough a SPSS program which applied t<]test, chi squ^^^ 

* "^^^^'^ 

^ *****W*******************^*^******* ********************* 



Note: A sample fJf^gfaAx'may be seen iri Appendix B, - ^ ^ 

'f . 

^MsHiual Tabulation and Computation of Statistics , 

■- M ' '. 

It is npt difficult with the help of ^ pocket calculator^ti^at has thernormal arithmetic f 
as well as square root to compute the statistics iC a computer is noV available. An excellent 
assistance-would be a copyof Bnining,-James L. and Kititz, B.L.,' Compuiaitonrf/ Handbook 
Statistics, 2nd qd-, Glenview, IL/Scott, Foresman, 1977. Thelntep by step method "can be 
by the novice without difficulty. The novice would be wise to have the figures checked by 
who kndws something "about statistics, , 



* bniwing Cbndu8iom (Generalizability) 

Results are only as-valid^as the care taken in planning, colleoting, and computing the data. If 
there is tiifte {md funds, a second project will be of value in supporting the fin^gs. Care must be 
taken to generalize the findings only back to the populations from which the sample is drawn, i» e., 
the findings in one^chooKdo not generalise to all students in the district, the state, or the nation - 
only tp the school from which those Students were drawn. y ^ > 

^Liesener Reconsidered 

■* 

A*number of aspects of the Liesner technique are not quantitative but do contribute to a more " ^ 
qualitative assessment. Liesenej^s usage of extern^ input from users iri his Step 3, "Determination 
of ^yvice preferenfces and priorities in relation to local n^eds," and later m his'technique where he 
reports back to the user what changes can be made in program (Step 7: communication of preferred . 
services currently feasible to total client group) is an attempt to achieve consensus in the evaluation 



process rather than media specialits trying to create ^change alone. 



Participants have rarely communicated as closely with user groups as lieseif^r 
recommends and so wilfwant to discuss how this nii^t come about in their 
partieularsituations. 



PSES as an Asses^enrlnstxumejat ' 



iinstxume 



Since Dr. Loertscher is*the co-author of the Purdue Self Evaluation System for School Media 
Centers (PSES), an mtroductron was given which surajnarized two research projects: 

1. Loertscher, David V., and Land, Phyllis, "An Empirical Study of Media Services in 
Indiana Elementary Schools,,'* School Media Quarterly, Vol. 4, Fall, 1975, pp. 8-18. . 

* 2. Stroud, Janet Gossard. Evaluation of Media Center Services by Media Staff Teachers 
• and Students in Indiana Middle and Junior High Schools, Ph.D. dissertation. W. 

^ Lafayette, IN, Purdue University, 1976'. ' ^ , "~~ 

PSES is a survey of teachers, ^udents, administrators and media staff which compares the perceptio 
pf.these groups on numerous servicerstatements in nine seI^dce categories. Each statement is rated on 
a frequency of occurance scale.(fl^Sa3y, Occasionally, rarely or never, don*t know) . 

^ ■ \ 4 - 



* Participants-were given computer printout containing actual data obtained from a * 
I school Which had used the PSES technique. Participant^ were required^o analyze * 

the data carefully, draw conclOsioris about Ael media center services in their school ^ 
and draw up suggestions for improvement. ^ 

* Co^ie^of computerized d^a reppfts that can be used in other workshops can be 
obfairied by writing " Hi Willow^search and Publishing • ^ 



* 



Box 2243 
» Idaho Falls, Idaho 83401 



♦♦♦♦*^****#*************************************** 



Three techniques are used to analyze PSES data: ; 

1. Comparison of user group means for each service to compare the overlap of 
perception. ^ 



vie\ f of services 





% of agreement 



Teacher view o£ 
services 



2. Gr^hing services frequencies for an "at a glance" look for problem areas. 

3. Analysis of which segments of users are being reached and which are not'. 

__. i • » 

The "so what" test is applied to each of the,three techniques. Participants find it easier to plan a 
program of improvement for someond else before tKey attack one of theif own. 

. ^ CONCLUDmG REMARKS ON STATISTICAL PROCEDURES 

' ^ " ^ (A°REVIEW) ' • \ ^ 

Using descriptive statistics - ^ • 

When ^ving questionnaires. we can often just use %'s to Understand the data. 

"When I ask, the.librarianjhelps me find a book I'd like to rdad." 



SA 



u ■ 



D 



SD 



T- Test • requires two means v^hich can be tested: 

scores on tests, e.g., scores on a pretest and post-test 

number of trials to criterion (howm^ny times the student goes through the 
instruction package before he gets 100% on the exam) 

requires frequency counts which can fit into a matrix at least 2x2 





boy 


girl 










passed exam 








flunked exam 








• 



SA > A U 



SD 



Eic 



Control 



Number of stude 



' if not at least i observations 
in a oeD then combine some' 
cells as.shown at the right 



r of students who answered ^ i 

Statistical Significance vs« Practicil^dignificance 
Booktalks via TV vs. in person 



Number of books' ^ 











in person 


.TV 


before 


.35 


40 


after 


75* 


47 



43 



ERJC 



- 38 - 



Using hand tally method: =^.6 df is 1 c,v. is 3.8 therefoi;e sig. at .05 using computer, it 
pripts the sig. level a^ .00265 " . 

•4 •' . 

Conclusion: something is significant but what? Inspect the cells to find the imbalance in the 
cells and interpret. ' . * ^ ' '\ 



Practically significant? compute: 



' ^materials costs ^ for each method, then decide using best 
- time costs j professional judgement 

' SQUIRT vs. no SQUIRJ - nieasiflred by the number of hours students read ^ 



Nmnbdr of hours 
daily 

Mean time 



group one 
SqiRT 


group two 
NO SQUIRT 




/ 


45 mih. 


25 min. 



Using hand tally mediod T is 6.85, c.v. is 3.45 therefore sig. at .05 using computer the sig. level 
is .049 ' - 

'Conclusion: Something is sig. but what? Insp^t means. Squirt the best. 
Practically ^jgnificant? Compute cost o^quirt, measure against judged v^ues and then decide, ^ 
. Siiiximary Goimnents on Stunpliog ^ 



Sampling in a school \ 
•Piroblems of sampling within a class 
1,. Sample size too sm^ ' * 

2. Too much interaction between experimental and control group. 



'■J 
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>^ Probable solution; "Sample Classes 

1. Determine number of sections (heterogeneous groups) 

. %. 

2. Are claises biased on treatment variable already? Do not use. 

3. Randomly assign classes io treatment. 

4. One third to one half classes sufficient for questionnaires. 



5. For homogeneous group's: _get equal number at each level 
sample individuals if you must. * " 

Generalizability • * i - . ^ 

Findings can only be generalized from the sample to the population of [which it is a part. 



■ 4 



-0 



J 




APPENDIX A 
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.1 



V 



I 



I- 



\ 
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lacerials 



41 

» New Jersey ^ ^.-"'^ 

V 



Present .Achievement in 
) local school 



•^ooks 



Magazines 

elementary scJlqio.I (K-6) 



Elementary school (K-8) 
Junior high' school 
Secondary school 
All schools 



At le'ast; 6000*10,000 titles 
representing 10,000 volumes 
or 20^volumes per student, 
whichever is greater 



40-50 titles (includes some 
adult non-professional 
periodicals 
50-75 titles 
100-125 titles 
125-175 titles • 
Tn addition: necessary 
magazine indexes and 
duplication of titles 
indexes, as required 




Newspapers 
Elementary school 
Junior high school 
Secondary school 
All schools 



3-6 titles 
6-1.0 titles 
6-10 titles 
One locals'' one state, and 
one national neyspaper to 
be represented in the 
collection'*^""'' 



J^amphlets, clippings, 
and miscellaneous 
materials' 
(vertical file) 



Pamphlets, government 
documents, catalogs of 
colleges and technical ^ 
schools, vocational 
information, clippings, 
and other materials 
appropriate to the curriculum 



and for other interests of 1 



students 



4? 



Films trips 



500-1000 titles, representing 
"^1500 prints or 3 prints per 
puplf*, whichever Is greater 
(the number ot'titles^to be ^\ 
increased in^ larger collections) 



.Son films 
Single concept. 
Regutat length 



1 1/2 films per student with at 
leapt 500 titles supplemented 
by duplicates 



S«ct. Oiit • 3 



42 - 



SERVICES-SELECTION POLICY 



A. A wrtttm Mtmnt hm bttn cl«vtlop«dClind acctptad by the 

£«! board to rafl«ct bacic foalt and..o^i«9tivM, to allow for 
handting of complaints, and tdlaiaura q<ij|lity aalectlon of a 
Kyof macarMf andaqutpmant. ' 



B. 



C. A policy i W imwt hat bMn drafted in gtoifal terms and does 
not yMd itatif to ipacific interpretation. 



E. No policy itatenntnt governing lelection exfstt. 



□ 

□ 
□ 




1 . It there an exirting written policy jtatement? 
Has it beea revised within thie past 1 2 months? 



Is a copy of the policy statement on file within the 
administrative office? " - 



4. Do procMures allow for immediate attention to 
complaints?' 

p * ' . . ^ 

5. Are forms available^pr registering complaints? 

» 

6. Are thare writ|pn criteria to assure quality selection 
of various forms of media? , ^ 



/ 



Nebraska Guide 



I 



Y0i 

Yes 

Yes 

Yes 
Yes 

"t 

Yes 



No 
No 

Nq' 

No 

No^- 

No 
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MEDIA «*jN0<9IRAM BUOQET SUMMARY 
Chtdk ^ fht foltowinf: Buildlni L«vt<. 



SyftMn iudf^. 



Sudlll (fvr nitdlt prafrim only) • - . 

tvm tfii CDftwt fifurfi in ooiumra V &nd Vaf tht chart b«(ow to tfiow actuil 
. MptmillMiii for tht modii profnm in^^in^K) ywri proctdinf tht currtm yttr,- 
and ki tolumn 'c* to ihowjht tmournt bud^tttd^ aptcif ic oataiorita durtm 
tTit eurram yaar. Spaeify yaart in all thraa < 




Itam(t) of axpandKura 



A. Una itam totali* 

1 • Library books and (natarialt 
tina 64 2.2 « * v 



Audtoviiual matariali 
lina 56 2.2 -11 . 



3. ti^ifqrnawlibn^iai 
tlnalOO^ 2.7 -4 . 



4. Fumiturt and equipmtnt 
(not replacement) 

Una 101 27-5 j 
(Report AVaquipmafH only) 



6. TOTAL of lines 
Aova 



1,^3, and 4 



6. Par pupil expenditure for mMla . 
program materials and equipment. 
Divide lina 5 above with the AOM 



IL f^EV^NUE (other than local funded 
A. State Dollars . 



B. Federal dollare (total of all funds 
used for media program) 
Specify soun:e by Title 



a Other: Specify 



Two^yaars 
preceding 

currant yr. 
197 - 197 



One 
praoading 

current yr 
197 -197 



currant yr. 
197 ♦ 197 



r 



— 



'Line numbtrt; thteL and corresponding codes have been taken from the Annua/ Finance 
Raport of fMfmka Public School 0«<r/ces_(Fpi[m AFR). 

••Tht term Hbrirlei ueed in lin^! 3 it broadened in this guide to encompass all enedia 
for n«w fidlitiet. 
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c 
E 



C 

e 

3 



a 

I' 

I 



2€ 

ui 2 * 
D 



1 



5 >> 

Ui lil 



«lt6m(s) listed by specific category 



A. Material^:' 
1 ./Books 
a. Hardback 



b. PapertMck 



2. Encyciopedias/fefirence 



3. Professional collection 



4. Per iodicars (subscriptions) 



&. t6mm and 8mm«film 



6. Filrmtrip>/slidts/|ranspafendit 



7. Recording: au(So> tape, di*<c 



g. Videotapes/disc 



Mapt, globes, charu, graphs, etc. 



U). Microforms. 



, . .3 ^ ; ,11. Supplies* ^.^ ^ ' 




C, Other (Sf^ify ifUbt listed below) 
1. Professional travel : ^ 



2. I nrtitQtionat memberships 



• t 



3. Professional gyvyth (Example: workshops) 



4. Other 



*0. fotAL^f^above- 



PER l^UPIL expenditiAre for rr^dia 
(Divide line *D' by ADM.) 



Two years 
Receding 

current yr. 
187 ' W 



Ooeyear 
pneceding 

current yr. 

JSLiM. 



current 
yaar 

187 ■ 197 



45 




SKt. On* 



Mf DIA tERVtCIt PHtf ILI«» 



A 




T« Mlop tttt MEDIA SERVtCES PROFILE for tht rm<Ki profram Mtm 
MrpMiritiefi, mm In «Hr1ppfopriM be^ M tfM dim btkm tht r«tln| (A, B. C. 0. or 
t) for Msh imdit arvio rtoordid on tht tdp of toeh of iht prwidint ptftt In this 
Sietion. For iMm#lt, If tht ttivict ntlni for SELECTION POLICY on Ptft 3 It C. an 
")(" tfraiiM bt pliMi In tht C cohimn of tht chtrt on tht hqfizontti lint It bt l td 
'"SMtethin Mey", Aflor ail twtlvt tHviott havt bttn rattd and a^'tntry madt for tMh 
larvloa in |ht^ chart, oennMt tht "xV «4tti a htavy tint to (»mplttt tht MEDIA 
4{ERVICES PROFILE. 



* « 

« 


Mwlla Sarvio^Platint 


A 




- I 




E 


ailiCUOn policy ^ 












Miction 0rociduffi 


o 










» Olttribution V 












AociMibility 












Orgmlzatfoh 












Imifvici imtniction 












Oirict tMching 










1, 1 


Confultitive lervictt 












Mat^ials and •quipment support 












Production , 












Oltttmlnttlon 
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June/1974 



. ft JW$ / MI ^VAlUATiyf OWW-jk ! - 46 -- » ^ - 



• 










T<(talHo«f» ! 




























C. Indicato tht diitrltatkm of mponiibltitlM f(>r>»rninf iiUdii^ mrrkm tamong oth«r proftMionml and iicmprofet- 
. ftonal Mmbiirs of ^ tawmlBf madia ataff. ^ ^ * 



D. Da all membars of tha iMumiaf media itaff (iKjth profesalcmal and nonprofawional) hayt apadal training in thair 
areaa'of rai^naibUiiy? Sptcjify tha extant and raeencr of the special training. ' ' 



E. €ive evident that.the leartilnir medlel center has adequate space, equipment, materials, and budget If there are 
Insdequaciee, indicate below the areji^ ^ grettest or most immediate need*: ^ 

h Space (such as areas for reading, Viewing, listening, instruction, individual study, shelving, storage of mate; 
rialir processing ^d prpduct^on ofl materials, and ofli^ for personnel). ^ 



. 2. Equlpmeht («uch as proJectorB-r-ii9|)tIon picture, overhead, opaque, slide, and fi!in»trip— scrwns,. t*pe recorders, 
, TV 8«t«, record playera, radios, cop^ njachlnes, study camjs, and materials production' tools) . 

■D?^^ ^ ■ I 52 • 



• JiJo/l^S EVALbAllVi; CKITLRIA^ ' ' 

. ' " * ' - SrcrfON VIII • UARNING MEDIA SfKV/Cf^ 93 

' ^* ^^mH^ Hir**' "•*Wt.. film., -fllmrtrip,. «lm Iood., aUm, Up«i. r^rd.. tAnW 

r:u'^ru^orri^5i:!;n^^^^^^ --^-^^^ «^rruction'^.t,r. 



i 



F. Wh«t reoonta^M, ktpt to faidicte th« um 6f th. Wuipmwt and mteri.!. lij,tl«^n,fa,jn«di« c«,Ur7- 



indi- 



H,If the lawnlnt mrfU eenter open to irtud«nt« and When at other than refular achool hours? Explain. 



ERIC 



.jj, ^ Cc-ntfruM on next paa".. 



. \ ]KSm^Km^^Z CRITERIA 

' SiatON VHI • UA»HNG,MtDIA Sti 

, ' • .- 48 -- * 

S. M«t«r{«b (iueh m boob, mftfuinM, ii«wq»|wn, fllow, lUautri|w; film iooys. dtdM, Ufm, ^"^4 
•nciM, pictiirti,. charts. Mp«, flobM, p*p«riMdii, eonnunitjr mourc* ftto, yntnmti iMtnwtioi 
y •. md V|i —j|il awtorUU for th* profoMiona] Ubrary). 




4. Budfet (*d«<u»te to fuHW roquMts pt itudMta a»l atiUr, ytwviil* jiennal r«piM«aMt m4 ta^rvvMi 
for toDovatlmi, m4 MMt .McrMliUnr «t«icy atoa^M^). 



F. What rMor«a am btft «» Mi«a*a tiM mm of th* avuipiMat ud Mtlnriak i« ilw |«mtaff MaMfa^T 



5 



G. Do tiM roMria akavpaMMTMar t>Mia ia Hkt i » i|iiia ij a<»aaal faeOHiaa, a«Ml»Mwl, aai mlarWaf 
caMtlMiralgaltaMMafcrfatMrafbapaiiiff. , ^ ■7^' 



' ----- - 

* t 



> 



1^ 



; H. la tha laamfaHT aM«« aairtar apaa «a aliUUata aa4 taMhara at tMkmr tkaa rafakr aakool haaraT B]^a. 



' * California 

. yi. >fedia Program Budget 
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In Part A, show actual media* program expenditures for the past three yekra plusyMidgeted ^ 
ambunts for the cumnt year. * m \ 




A« Lo&al funds 



; . 1 , Audiovisual equipment (hardware) 
2. Audiovisual materials (software) 
.3. Audiovisual replacement 
4. Contract - audiovisual 
5! Contract - library 
6/Filmrefital 

/It*' 
. ^ ' 

.7. Telecoiiununications v^^t^- ' / 



SrUbnuy books 



Three years 
ago 


Two years 


Ust 
year 


Thii 
year ^ 


• ^. 






















S 

























4 



















r 



M#dia Program^ Budget (Cont.) 
At Local funds (Gont.) 
-9. UbraSy furniture . 

10. Peripdicak - subscriptions 

11. Textbooks 

. ilSupplies ^ ^ ^ ' 

1 3. Building facility modification (dea^bt) 

14, Other J ; ^ 



ERIC 



B. Total fcdeml funds received 
* u| the foltfwiRf ii«-o|reim 

ESEA^Titlel^ 

EiSEA, Title n ^ 

ESEA,^itle III 

NDEX, Title III 
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Three years 
ago 


Two yt are 
ago 


Uit 
y«ar 


. Tlvfs 

y w . 












4 


















* 












\ 




^ 


I ^ 




4 


' I ' ' — ' 














1 * 

* 

1. - tiT 








* o 


t 

> 










*> 

'S 








t 

1 — .J 



\ 
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VIII. Media Program Inventory (C6nt.) 
B. Facilities (Cont.) ^ 



1 1 . Story telling afea 



12. Other (Ust) 



C Instructional materials 

« *^ 

r r 

1. Art prints 

2. Art objects ^ 
' 3^ Book$ (Exdude alHextbooks.) 

« • Sets of encyclopedias and other referenoe 

b. Hardbound 

c. Pap4^rt>ack (an types) 

d. Frofessional^ \ 

4. Charts 

5. Cuhiculum f uides 

6. Dioramu. ' 
\ TrDisc recordinp ^y^'^ ^ 

8. Documents 

9. Fams^ 
a. '8mm cartridft 

' ' (1) Silent ' 



i 



(2) Sound ( 
b. 8nun re«I-to-reel' 

a)saeiit 

(2) Soumd 



Indicate 
square . 
footage- — • 




Inadequate 












V 










Number on hand 


District 
IMC 


Buildini; 
IMC 


Total 




• 


-t ^— 






\ 




— \ • 

c 




























t 

<* 


- 










r 


' % 
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California 
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QUALITATIVE EVALUATION CHECKLIST 



Pactcr t« bt mknM 



1 , AppropriateMfli of objectives of media profmrn to district 
piuloaophy tno oojectivet 


1 


1 


1 




1 


1 




1 




1 


1 


1 


1 


1 


1 


1 


.... . ■■■ •"' ' 
2. Adminktrative suppod and commitment to media program 

' P •- 


































3. FacOitie^ * * » 




• 






























4. Penonnel (numericiny) ^ ^ 


t 


-> 












« 


















a« central oitice 


































c» D. ^DUiiainf level 






• 




























S . Competence of prof^ional perionnel 


















r 


• 














a.^ \^enirai oiiice * • 
































• 


V, 

. b. Building level 


































■ c ' 

xj*. iiipuuciiuiisi nuiicriais < 


4 
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! 


; a. vuanuiy 






































• 






























/. cqyipmeni 






























£-1 




a. v^anuiy , 






























\ 




D, vuaiiiy 


































o". DUdget • • ^ , 


































9. Dtstnbubon system 








f 


• 
























a.' A'ccessibilitv of materials . ' 


































i 

b. Accessibility of equipment ^ * ^ 


























/ 








— ^ — , — f — • — — 

.10. Involvement of teaqhers m selection oi matcnals 


























1 1 . invoivemeni oi siuQcnis in seicciion oi mHicnais 


















1 
















12. Involvement of media personnel in curriculum decisions 






E — 


















13. Provision fot inservice training activities 






























* = 1— 

14. Staff effejCtiveness ^ . . 






























• 




i — •— • 7 r — 

!!5. Past evaluiftion procedures 
















i- 


















1 6. Achieveoicnt of program oBjectives ^ 


7 

































> 




\ 




* * 


• 


A. 


fflimixmrj «fTmlttiitioa of .th« fchooX libr%i^7^ob^t vras ^ 

Thl« •raluatiaa iu m i«it«t' - aao"«od foras Jtie bUp la tiM •»»lu«tlon of the achool litfrirj, 
Tb.9 rsml fimaatiaa (••• C pt«f 10) is to b« B»&e «f t«r all ffeeta hsr* b«fla eatherad «nd ell dveklua- 
iiona a*d* in Gtaptm 2 U ^. • % 


• 


• * 






the obj«c^'T« hM bMa achlinr«& anf. the iaportflne* attaelMd to tb* obJactiT*. £atinatln« the 
decree of succMt on, b« onlgr « •ttb>ctiT« appraiMl siaoe pr^Gi— tmchaiqa— for ■e^unofi-thie 
fku. do not «SFi«t, Ow ooA* fcr tlie ajMbola to b9 «Md ia this snniisal is as foUotira: 




Q 
a 


p 








t 






A - Ii:o#lliat 1 ^ Ot uteost Saportance ./ ^ 
1 - Good . ^ 2 - Cf ociMiid«mUlo Saporiacce 
C . Fair" • -5 • Of li^port^ca 7 " - . 
!),• Poop , ' . ■ t , * 
' 'r- »ot at all * . " / \ 




§• 
33 

CO 
O 


« 




T^tgTM of mccmJ* 
In achiirnat 


Ob^tlTM of tiM^achooX Ubrarr 


ifetlMfi new boing uaed b^jtha llbnt- 
riatt/to aehlara ob^tiraa: typea of 
Xibr^ aarrioa, fctiTlt^ieB vith 
atttdaata aod taaehera. etc« ^ 




8 

■■- v. 






ABCD.r ^ I - 
123 








:? 












o 

Q) - * * 








: *■ ^ ■ - 


' • .. - ^ - 

— ^ - - * , 


V 






« 


A 1 C D f 

12 3 ' 






# 


♦ 


* ♦ 




A B C D T 




* 










I -2 3 


* • 


*.* 


• 


e 
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♦ 




^ . • . - - ' 
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I. INSTITUTIONAL, EDUCATIONAL MSDU SBKIflCES 



CRITEm 



^An institution ihould have a pirogram of aedia tarviQat administered through 
^ an educational media center^ and sub^^centars if such are needed » which 
provide the faculty with a;» adequate supply of appropifiata instructional 
materials* / ' m ' * 

' ' ^* - /■ • J . ^ ' . ' 

**Tha educational media^center should be a sapar it e' serv ice iinit that^ operates 
at the saa^ level as other iuijor ins^tut^onal servicasr / , / ^ 

^"'1^^^^^^^**** should have clearly defined policies, .pjrocedures^ and 
,>_pAani for its adutatioflfal media progr air including short-range and long- 
-range goals* ] • / - 

There ahould be m sufficieut-^lua|i*er of professional media ataff .mcd^ers to 
administer the educational madia- program and to provide consultatiVf 
services to am iastitutioin^s entire faculty. 



Commitmaat to the. Madia trpgtiii* 

The institution's educational madia program does not offer the 
g services of a media center aAd mo clericatT or technical staff 

^ ^ ^ ^ m imhira are available to administer the educational madia pro- 

2 Q Q O institution's aducationaV vadia program consiita of media 

• services from a media center managed by clerical and technical 

u " staff * ma mb s ra* 'Thfe aervtcea are not well coordinated and no one 

«M . - person haa^been given .ailministrative responsibility for institution- 

^ wide madia activities. 

e , . . - 

The institution's educational media program cona^ists of a media 
J} ^ center with clerical and technical staff. The program is directed 

S ^ staff person who has some media training but not enough to^ 

"m\ qualify him as an educational media specialiat. He reports to 

the institutional administrator directly respbi&sible for instruc- 
tion. ' - ' • % ' . * 



Eg o 



The institution has an educational media program including a media^ 
center and necessary sub-centers directed by an educational media 
specialist who repots directly tct the adiiinii^rative officer in 
.charge of instruction, le l§_jfta^ with facilities, finances, 
and staff essential in jmieting tbe iladia needa of the instructional 
program. . , 



Fulton 
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PROFILE SHEET 



To develop a Profile laege of your program^ transfer your mark from each 



.Itea^of the Ev< 
straight lines 
torlallj^ deaoni 




iluatlye Checklist to this sheet* Connect the marked squares, by 
» Then turn the sheet to a horleontal position* This wlLi pic- 
itrate thei"peaks^ and **valleys" of attainment for your program. 

WEAK '[ STRONG 

Nark only one of twelve boxes 

□ 3 □ □ □ : C3 □ □ cl © II 

□ □ d.Q . □ □ □ □ Q O p *@ 





□ □ 

□ □ 

□ XI □ 

o □ □ 

□ □ □ 
b a. a 
a □ □ 
□.□ □ 

□ □ □ 

□ □ □ 

□ □ o 

□ □ 

□ □ C3 



Q a □ 

□ cavQ 
a □ □ 

Q □. □ 

a p □ 

□i □ 

a □ a 

p □ □ 

.□ □ 

□ □ □ 



□ a 

□ □ 

□ □ 

□ □ 

ti a 

□ (3 

□ a 
□ 

□ a 

□ p 



sip O U 



□ Oi O 

□ a m) o 

13 la s 

a s 0 0^ 

□ @ o o 

Q O 19 



□• a □ □ □ IS 13 0 



a □ □ □ a 

(3 □ □ (3 

(3 □ 0 □ 



ti m B u 

•(01 El 
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0' in 
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^ PROFILE fc^UDGET 13 



MEDIA PROGRAM BUDGET NUMMARY 



Building 
District ^ - 



BUDGET ffor media equipmenf, materials, personnel, and serviceti 
Enter the correct figures in column*, "a" and "b" of .the chart below to show actual ex- 
S^wi?'?? "^^l^ program in the two years preceding the cuirent year, aod in 

column - c^ to |how the amounts budgeted for specific categoriM. during the current 
year. Specify years in all three columns. • 



ERIC 



I. Item(s) of expenditure 

* 


1 ■ — * 

2^yeara ^ 

current year 
♦ 197_-197_ 


1 y«tr 
preceding 
d^urrent year 
,/ ld7_ -197« 


Cutrent ' 
year 

197_.1^)7_ 


> a 




' c 


A. Line item totals 
' 1. Library books ^* materials . 




$ 


i 


2. Audiovisual material's 






C 


3.* AV equipment (not replacement) 








4 * AV equipment <repjarement) 








5. TOTAL of lines 1,2, 3, It 4 - 
ateve 


5 — ' 

c 


f 




1. pupM expenditure for media 
program materikis ii equipment. 
Divide line 5 by the enrollment. 








\C. Media Siaff Salaries 




1 

1 




*Nole: iMilude not only portaMe television equipment, but also ceiitraliied teievii 

. — V 


lion iiutAllations^ ^ 



A. Total district budget 
1. State funding 



2. Local funding' 



3. Federal fiinding 



1. JMia budget 

V 1. i)iaUict 



1 Federal (Identify) 
a. 




c. 



3. OUier(PTA:etc.) 



V 

J 
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M-:i{VHl:s AECT 56 . ^ \ 

-Servicef: Selection Policy— II 



^3 - 



[ J A. An approved 'flUtement is on file which eitablishei procedures for handling 
challenges to materials on the basis of content: 



1 B. , 



U.q, A policy statement has been drafted m gXMn) tems and doaa not yMd itadf 

to specific interpretation. ^ » 



□ D. . 

□ E. No policy aUtement MiaU. 

\ 

CUMMKNT: 



CoMliaiiitji le A (afeMMc) Capcul* piopoHla to nMt A 
if PMUitiaa- ' ■ ' - * « 



O — ■* 



9. Fao^- 



r 



4. PoKcy. 



5. TfaiM- 
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. Services: Reference 



□ A. The media program jprqvides.thB expertise in information retrieval service to staff 
and students in support of ihstrucUonal objectives. c-^. ^ 



□ B. 



□ C. Reference materials are available to staff and students for seeking out information 
but there is no service rendered to assist them in the compilation of information. 



□ D. 



I- 



□ E. There^is no provision for reference service. • j \ 

\ : . . 


CO^|MENT: ' ' 






1 

/ , 


• 




Constraints to A (above) Capsule propolis to me«t A 






1. FacilitiM--" ' ^' • * 

0 












2. Money- < ^ % * . 






3. People- 


• 




4. Policy^ f, \ 







5. Ttm«- 



/ 
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■ -Part-2^Media Duty Profile 



Instcuctioiu for uie 



Part 2 of'this secfcion IS entitled the Media Duty Profile and is intended-te-indicate 
tne inyolvement of media personnel in various assigned duties. The list of 30 tasks is onlv 
a sample. For additional items, see the note at the top of the following page. 

uVfhpE^f"^T*" ^^'"i is served through a «ngle professional person, 

use the letter "c" to signify that media specialUt. _ 

Example: 7. Instruct^ students in media skills. A B j^D E F G H 

Circle the letter or letters following each sUtement to identify the person(s) who p«.rform 
?!* J" f-.Ti**' «8«iKnment at the school level. Underline clearly in red (or other 
color) to Identify the personfs) who perform each task at Xhe disfUct level. 

Example: 14. ProducM fraErfJic,^photographtc «nd * ^ ^ 

audio ingtrucffonal materials and 

displays. ' % (cjjg^ E f ^G H 



MmHm Duty rMAIt 



NOTE. Th€ item^ below ore Uluafjrmtiue only. ^0{ker^dr tdditioiul Uem$ may be used 
Commit the Beki^iotml RequiremenU Anely$i$ Chrckli$t (ALA) tndfor Jobt in Imtruc 
Hontl Medim Stutfy (AECT) for item*. 



M«^ia Staffing referenct>s in these^ pages are as follows: 
A Dtractor of dislrict media prdgn^ 

• Head of school media program ^::=::=:,Certified \ 

C Media specialists ' : ^ ' 

D ^edia technician^ ^ - - ' 

Media aide andy^r clerk \ * 

F Student aide " ^^-.i Noncertified support personnel 

G Other . / ' i r~ r , 

N Not Pe rforme d X 

Please circle the letter or letter following each statement to identify the porM>n(s) who ' 
perform each duty as a maj6r assignment at the school level. Underline in red (or other 
color) the letter or lett4»rs following each statement to identify person(s) who pt»rform 
each duty as a major assignment at the district leveT 

, > • • 

1. Confen with admiiiiUmtors snd/or «ohAoi board 
' poiiceniinc media opc^rUion, profnmw and bud- 

•^•^ •• •^ ' A B C O' E r G H 

2. ^irtictl^s in cunriculum devWopmeiit aii'd 

/ . A B D E r G' H 

3. fW^igriK U*aminie materials . . ^ B C O E F G H 

1. Help* to develop and implem4»nt propoMris for 

fedrml prnjiK'tft. priigrsmK Slid service units ABCDE FGH 

- " * • 
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AECX 



'ro7a( ('({llection 
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a. Base <u)Uection in the 
school 



l>. Basic recommendation— 
state or local (convert to 
repretent actual school 



•ize) 



c. Nwd ("b" minua "a") 



d. Rating ("a" dividMl by 



e. 1975 AECT/AASLif««a 
Pro§rmm: DMriet S 
. School reconuMAdatioM 



f. Need ("«" minus "a") to 
1975 NcoMitMndatioiis 



JL 



K. Acquisition plan to reach 
goal ("b" or "*" dbovt) 



-MATERIAL8- 



EQUIPMENT 



in th^ Mchdol or 40 
itMii p«r milr. 



Sh^lvinf and/or cabmtts 
to accommodttf a aiini- 
mum of 40.itMM par stu- 
dent. axdiMiM of^ taxt- 
books. 



19 



(*ongider- 



If 



If 




Urn 












- tMal 




5,000 tltl«M fwr l-<'lwttfit< Cm^mtt 
(MoXUfX* «opi— f ■ t ia«T 














I 1y, 

^^^^i€a) p4r K-a liawtt Castar 






f 




o 


5tudMV^Prof«B0ional 
.Bodftt #^2«32290 


l'5-20.KrB Includdaf yrof ■■■1 natl jw^Tpiwi 

for K-6' . : 












PtofMSlCMl 

Hm^lrtw— and books 
Budgtt #502.3250O 


K-B allotMnt ^ ^ ^ / 
SpadjOlaad jtelar^ki^i la Bil lajjit ^ 

■ ' ' < ^ . © ' '"It 

* 4 




• 

** 








Budget #502^?&290 


2 tltlaa par K«6 liaiin Caatar ^ 

' 3 tltifa par'^tf^ariailt laaaartt Cmnkm . 

^ # ft , 

' .V. i • ' 


• 




« ( 

1 
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APPENDIX B *7 ' ' ^ 

' ' ' . ■ 



.. > ' . ! - 

HYPOTHESIS: ) 

There will be a significant^fference between the number of books read by students 
y ^ prior to ami following a book talk presented orally by the librarian, ' • 

Subjects were 30 students m a seventh grade readmg class consistirfg" of an equal number 
^ of boys and girls ranging in I. Q. from low to hi^. (S^e tables.) ^ >^ 



t 



/ 



c 



BOOKTALK 






ABSOLUTE 

FrEo 


' RFLaTTVE 
FrEO 
(pCT) 


ADJUSTED 
FREfr 
(PCT) 


CUM 
FREO 

(P'CT) 






lOOiO 


• 100,0 


100,0 


TflTAL 


10 


lOOtO 


"lioo.o 





MEAN 
MODE 
RANGE 



r,OOQ 
l.OOo 
O.OOa^ 



STD ERR 

STD DEV 
MINIMUM 



OjOOO 
0,000 
1,000 



MEDIAN 

VARIANCE 

MAXIMUM 



1,000 
0,000 
1,000 



4 



Subjects =^ 30 
(30 participants) 



VALID CASES 


So 


MJSSlNfi 


CASES 0 


1 






- 


BlSbiCTALK 














29»JUl-7<r 


Ftl,! NONAMF (CiEATtON DATE « 


29«jULi74) 








10* Z 
CATfftORY IfiBri 


coIje 


RELATIVE 
ABSOLUTE F|,Fq*. 
FREO (pCT) 


ADJUSTED 

FrEo 
(PCT) 


CUM j, 

freq * , ; 

(PCT) ' j 






High I.Q. 


I. 


6 • 


20,0 • 


20,0 . 








Average*!. Q 
Low I.Q.. 


^ '3. 


IS 
9 ^ 


5fl,0 

i6o,Q 


90,0 
30,0 


\ I 

70,0 ' I 
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EVALUATION TECHNIQUES, FOR SCHOOL UBRARY/MEDL\ PROGRAMS 

ABIBUOGRAPHY ^ . 

' Compiled by David VTEoertscher and Blanche \Voolls for an institute conducted under a grant from 
the U.S. Office of Educatioii/litle IIB, Higher Education Act of 1965, P.)L. 89-329, as amended. 

. INSTRUMENTS 

^ ^ ^ ^ ' ^ * • . ^ 

Association for Educational (^ommifiJ/e/ations artd Technology ,*Commj^tee, on evaluation of Media 
^Programs, Ev( ^uatinj^ Medi af Ingram/: Dtsiript and School; A Method and an Instmment The 
^ Association, 1976.. * . ' ^ , 

"This evaluation mstnunent is based gn the assumption that the purpose of evaluation ^ 
is not to^prove but to improve. This instrument^is intended for use as part of a formative 
evaluation process." Includes profile and budget, services, personnel, physical facilities, 
,^ 'Collection, and student and teacher opinionnaires. ' : ' 



Case, Robert, "eriteria of Excellence Checklist," School Libraries 18:43-46, Spring, 1969. 

;"The /Criteria of Excellence'iJwas first used by the School Lib rary> Manpower 

Project, funded by the Ki^p Foundation of North Carolina Inc;, as a guide fti 

identifying outstanding school jlibrar>' centers offering imified service programs at 

the building level,"* This checl^t ",wil1 provide the foundation for an-ih^ial ^ 

evaluation of individual schopl^library programs."- * ' J *' 4 

, D;eProspo,- Ernest R., and Samuels, Alan R'., "A Program Planning and Evaluation Self Instrudtional 
^Manu^" unpublished paper, L975. . ^ " " 



* ^ ©esiened as an "dh-going^effort,in library /ipedia program measurement and evaluation" 
• ^ ^ , 'this lujfiSshed manual proyides ^^strategy and tactics of program planning and evaluation . 
withinj,the context of the^^ool media tenter/* 

Disi^ct \m RegtonalLearmng Resource (Media) Pro-ams: A Systematic Planning Proce:ss ami \ 
zEljplofa^ry Survey pf^S^ices: Final Report. AustjjjLv^xas, Texas Education Agency, Division j 
o£Instig^r3lJle|ource^,1975.. * ' * •/ ^ • • , - ) 



^ 7^^? Liesener^ technique of planning as applied to .district and regionaT media 
prograu^s. r Plans-are to pubiish this report, buHt is not curre " ^ 



^ Fulton, W. R. ,and King, Kenneth L., Evaluative Checklist: An Instrument for Self-Evaluating 
an Educational Media Program in School Systems, Washington, D.C., Association for Educational 
Communication and Technology, 1970^^ 13 pp.' ♦ * V 

"The status o^an educational media program is hot likely to be known without 
periodic evaluation." This checklist was developed to aid'in such an evaluation and 
' . • ' provides-guidelkies fpr.making judgments on program elemehls. 



Gavfer, Mary Virginia, Effectiveness of Centrali2ied Library Service in Elementary Schools, New 
Brunswidk, N J., R,utgers University Press, 1963, 268pp. ^ - * . 

The evaluative instruments for this study were developed to assess the avail^^^ity 
of library sendees in elementary schools in terms of materials provided, resources 
'arifl>services, library related activities, pupil mastery ofdiArary skills, and reading 
dcffiG by childreq. Educational aohipvement was measiured as well as the ability 
to differentiate between schools with various depees eft libraries provided. ,C]|ieck- 
lists used for this study are given in the Appendices. 



Gaver, Mary, and Jones, Milbry-L., "Secondary Library Services: a Search for Essentials," 
^Teachers College Record, 68:200-210, J)Gccmhcr, 1966? 



Tliechecklist developed for this study was similar tq the one used by Gaver in 
Effectiveness ^f Centralized Library Service in Elementary Sdtools as listetfabove J 
this basic list of 1 It) services was developfrd through a survey of the literature qn | 
library .work with young people. , ^ ' 



Gaver, Mary VirgiiAi, Services of Secondary School Media^Centers: Evaluation and Development^ 
Chicago, Americ?ai Library Association, 1971, 131pp./ ' 

A report of a follow-up of ^ 1£ 65 study which was done in 1969. C|iecklist^f 
services lists theVariety in nuir ber and kiiijj; of services and thef balance>in-pre8enta- 
of services in all theirfts of the media center progranf? , • T . 



^Gaver, Mary V,, A Survey of the Educational Media Services of Calory Public Schools cdnducted 
on behalf of the Calgary School Board, Edmonton, Alberta, Schooiof Library Scienfce, University 
of Alberta, June, 197 1,' 137pp. mimebgr^h , ' 

\ - This i:eport of a survey of media "Services offered in ^Galgaiy cont^ns a survey* 
^ instrument. ^ ^ ' 
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Henne, Frances,, Ersted, |luth, Lohrer, Alice and others, Planning Guide for the High School' 
LiBrary Program, Chicago, American Library Association, 1951. 

^ • ■■■ r • 

A questioniiaire designed to ev^lluate the existiiij^ activities and services for students 
- and teachers as-well as the staff,, maferiafe collection, budget ind facilities.^'Airee 
* * classifications of responses were used: . How good? (To measure woSSi or condition); - 
To What extent? (To measure degree of frequency); and Yes or No? (To permit 
* <• variations in an&v*ering). The final chapter di^cusses^the developmentfof a plannihg 

' • program for the school. J • * ' • 

**An Instrumenjt for the Qualitative Ev^uation of Meida Programs i|j Caljfonlia," Sacramento, . 
California, California State Department of .Eaucation, 1972, 37pp. 
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^ California's metho^^-^asiessing the adequacy of media programs throu^out 
- California." * - 



Liesener, James .yf^lanning Instrum&nts for School Library Media Programs, Student Supply 
Store, University of Maryl^d, College Park, Maiyland 20742. * 



5 * To bemused with Liesener, listed bfelow. ' : ^ • . ^ - 



Liesener, James Ww', A Systematic Planrimg Process 'for Schopl Media Programs, Chicago, American 
IJbrary Association, IftlfiLLfiepp, ^ * ^ ' ^ ^ , ^ 

\t "The process, techniques, an<f conceptiial m^el . . . tp conceptualize media programs 

> morels tematfeally and^to 'develop more sofhisticated man^^ment tools fbr improving . 
\ th^i^pability of media pe^onn^o artrculat? and ^tevelcp mdre responsiv€"ind effective 
- /programs of media serviceC^' Preface,^v. ^" 



Loertscher, David .Vickers, MedictCenter Servi^s to Teaqhexs in Indiana Shior Hig%* Schools' 1972-73. 
Unpublished doctor's thesis, Blooniinton, Indiana, Graduate Library-^BcHdol, Jndi^ma University, 



1973,149pp.. - * . * '^ ^^ . K 



Two scales*are used fbr this/jhecklis^; The first evaluates how ^^gfc the &riace 
is*pro<dded and the second rates^ow satisfactorily tl^ s^r^ce ^^Haded. 

* \ Loerticl^r, Davfd V. and Stroud,Janct G., ^S£S:. PurUue Sj^^Evaluafion. System for SchooiMedia 
'CentSK: Elemental Catalog an^PS^; t . ; Junior, siniorMgivSchgol Catalog, botfc^Idaho Falls, 
* ^ 'idaho/Hi Willow Research and Pulilishingi 19?6, 24pp,\jL 
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•Catalogs of more thaji two hundred service j tatemeiif^^ wi^ch can bfe used to measure* ' 
"^eNawireness, frequency, ajid segment of f^ie us'^r gjToup^ho are utilizing hiedi^ center^ervices. ' 

" '"^ . ^* n • . ' it \ ' * / 



National Study of School Evaluation, Elementary, School Evaluative Criteria, Arlington, Virginia, 
The Committee, 1973, 152pp, f. ' . / 

Criteria for evaluatmg the elem^ary school which includes the media center. ' 

National Stu4y of'S,econdary School Evaluation, ^^twi/wa^tW Criteria for the Evaluation ofi 
Secondary Schooby^ 4th ed.; Washington- D.C., The Oommittee, 1969^ 356pp. . ^ 

Criteria developed for the evaluation- of secondary schools and including tl\c 
» , evaluation of the media center and services. 



National Study oj School Evaluation,^/wntor/%A School/Middle School Evaluative Criteria: ' 
A Guide for School Improvement,^ATlmgton, Virginia, Tl^e Committee, 1970, 152pp. 

Criteria for thejunior high/middle school. , i . ^. 



*trhe Nejbraska Guide for Establishing, Developing, Evaluating, School Media Programs/' Lincobi, 
Nebraska, Nebraska State Department of Education, 1^75, varr'pp^ • ' 

. Measurement of the success of instructional materials and accompanying . ' ^ 

equipment to meelTthe demands for both new and old approache^to education. ^ * 



New Jersey^, Department of JEducation,.Publit: and Sch^aol Library Servicer Bureau, "New Jersey 
Blueprint for School Media Programs," The DepartJnent, s.nV, 13pp. * 

^ * ^ 

- Definitions j^cede a checklist o¥ stiff, location and^space, collection - materials, \ 
, and equipment. ' Checklist designates a goaltand space Js left to tecord present " • ' * • 
achievement in the; local school. *l 

North Central Association of Colleges and^SchorJs; Ad Hoc Committee o^ Instructional Technology, 
Commission on Research and Sjsrvice, *^nveittory of Rolicy and Program foi; Ijistmctional Tech- 
nology," The Association, 1975. ' K ~ ' ' 



( 



Desigped for use in colleges, thi5 booklet is organized into two parts; 1) Policy 
and ProgramJnventory for Instructic^ol Technology and 2) Checklist of Specific 
f Items I^entifialSe witk Policy and Program. Bas^ically uses yes^no and observed/not 
observed responses to qvestions^ 



Peterson; Frederick WjOliam, TTie Developjnent of Evaluative Criteria Jor the Services Provided 
Local School Districts by the Area Media Center, Unp\iblish6d doctor's thesis,* Lincoln, Nebraska, 
The Graduate College, Univershyof Nebraska, 1969, 97pp. 

* ; T ^ ^ • 

. "This study developed evaluafeve, ijriteria-f or the afctivities of^the* area media ^ . 

center as these activities related to the local school district. Notjonly should 
thede evaluative criteria provide administrators and'media speciaHsts with irifdrma- 
* tion of value in planning new activities for area media centers bi/t should also * " 
provided measuring device for the on-going evaluation of area media centers." pp. 34. 

' \ ' - • ' ' ^ ■ 

Phillips, LuOuid^Viiison, "A Quick buf not Easy Test to Determine How You're Doing as a * * 
SchooliSbrarian or|<edia Specialist," Wilson Library Bulletin 50:399-401, January, 1976. 

A simple self evaluation instrument "designed to^elp you gain insight into 
you and how you see your job" - Very much.a selLapalysis-test. 

Southern Association of Colleges and Schools; "Evaluating the Elementary Schools Library 
Progr^," The Association, 1964, 26pp. 



Questions to be answered regsbrding overview, viewpoint, ffiUjctioft^, programs, 
SSiSjCRCH DESIGN AM)^^ . ' 



-resources planning in workbook style. 



jsiazek, Ron, Influencing, Stutientsimoard Media Cente^se: An Expertm^taUTii^stiga^ ' 
'Mathematics, Chicago, Ainerican Liturary Association, 1975, 176pp. " ; . * 

As a report of an experimental doctoral research project, this work "is intended 
to.aid individuals,^ school media specialists an4 teachers afike, determine for * 
^themselves the nattfreof theif potential contnfrutioh in f>roducii^ sttfdent users 
J of school media centers and libraries." Preface, p.xv. ^ 

Brunmg, Jame^., and Ki^tz, B.L., Computational Handbook of Statistics, ^irfiview. 111., Scott, 
Foresman md Co., 1968,/i269pp. ^ • v \ 

"Most statistics textbooks, concentrate oh thec^xctical discusaoJis/and mathematical 
proofsjOf'th^ various concepts pre/;entq3. Thefifesult of this^^p{jroach is that students 
. 'often have little uncierstanding of hpw.actually to apply statistical te^s to their 
fundSamental findings. The'iritent of this book is to reverb this ap(5roac|fond to : ' 
•present statistjica^concepts and iests^as they are applied. Eiom the Frefack * ^ 



CampbcU, Donald T., .and Stanley, Jylian ^^Experimental and Quasi-Experimental Designs '. 
/or i?«ea;j-c/i, Chicago, Rand McN^lljr, 1963,81pp. -- - * . ' 

An exaimmation of "the validity of 16 exp<rrimental designs against 12 comrfio^i • 
threats to valid inference^' ' 

^**' * ^ »* ♦ 

• f • . ' ' • 

. * * ' •> ^ » * 

• Case] Robert and LoWxey,Anna Maiy., Se'haviohi^eqi^rements Analysis Checklist: \a Compilaf 
■6f Competency Based Job Functions and -mk' Statements fpr School Library'Media-Per^nel. 
Chicago, Alherican Library Association, 1973, 60pp. ^ " " ' * 

. ; ^" ' ' ^ ' ^ .. ' . 

°f approximately TtJolksks to be' performed by schpol library " 
media specialists ... to anticipate, and in some instances .to conceptualize the functions 
■• - . / ^^^^ °^ school library media specialist.toWet the present demands and 

" •/ future needs,of school library media center us?li." 

I ' * • \ , *^ ^ I 

• diasf , Cfinton I., Elementary Statistical Procedures, 2nd ed.. New York, McGraw-Hill Book Co 
.:, 9.926, 27?pp. • • . , ■ ' ' • : ■ 

:>i*The qjlphasis is on deriving procedures out of logical 'structures that underline • ' 
the" -methods of analyzing data." Preface. <- " ' 

-\ ^ ■■ V *■ ' ' ^ ■ 

' "Cbmpetenpi^br-Speci^ists in Med^ Manageme~n"t,"^ Audiovisual Instruction 19 :30-44- 
Novei5*er,,1974\ ▼ . J' ^ ' , .' 

' Vl';^^'^';^^^^^ adapted^ froi^^^m^/iR^or^: fobs in Instructional.^^ 

•iic. ¥^'4A^t^^^ which can scTve as an'idea soijrGe for tasEs which may need evaluation. 

Encyclopedia 'a^L^Mc^^^ Francisco, Jossey-Bass, 1975, 500pp. 

* *' "\ %\ ' • , .f ^ . • " ^ . . 

' - ; Short ^cles describ'ing J?oth main concepts and techniques of evahiation. 
' Abibliqjraphy'is.appe^ , . • ' _ * 



Cuba, Egon G., and §tufflfebeam, Qaniel L., Evaluattdn: The Process of Stimulating, Aiding 
and Abetting Insightful Action, ^^I9)i0graph Series^in Reading Education, Bloomington, Indiana, 
Indiana UniveT|i*5y, June/ 19"70;^ • / . 

Th^au.th^^rs do npt believe "tha,t evaluation is equivalent to research . . . many 
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Isaac, Stciphcn, Handbook jn Research and Evaluation: A Collection of Princitles^ Methods, and 
Strategies Useful in the Planning, Design and Evaluation of Studies in Educatt^^ndthe Behavioral 
Sciences, Sai Diego, California, Robert R. Kn^pp, 1971, 186pp. " 

# - A J ' ' 

"The Handbook presents only highlights, outlines, and essentials to achieve emphasis, 

cltrity, and brevity. Each user "ft expected to sup^ment this document with more * 

'.j ' complete information from the standard texts or^ualified consultants in this field, 

• once he has lys bearings." p. iv. 

KerUnger, Fred N., Foundations of Behavioral Research, 2nd ed., New York, Hok, Rihehart and 
Winston, 19647 741pp. « ' . ' 

• One of the mos^eadable^ yet a^britative, of all statistics textbooks. - 



^ Questionnaires for Research: An Annotated Bibliography on Design, Construction and Use, 
^DA Forest Research Paper PNW440. Portland Oregon /Pacific Northwest Forest and Range 
^ , /Experiment Station, U.S. Department of Agriculture, 1972, 80pp. * 

* ^- , 

, . Fox: those doing mail questionnnaires, this publication summarizes studies which 
evaluated techniques for questionnaire construction and administration. 



Seibert, Ivan 1^1, Educational Tech^olgy: A Handbook of Standard Terminology and a Guide 
for Recording and Reporting Informati m about Educational Technology, Washington, D.C., 
U.S. Department ofHealth Educalion a|id Welfare, 1975, 276pp. 

m 

A section of definitions of evaluation ter mino logy and giridelines for jneasurement 
of collection si2e5 are included in this handbook designed to help people gather,* , ' 
compile, aiid interpret data relating to the application of technology to instructor. 

"Shapiro, Peter D., "After Data Collection: Coding-An Efiucati^nal Research Tool," Stanford, 
Califoyjfiia, Stanford University, Institute for Communicatio.n Research, 1972, 10pp. 

^* An analysis of ways and means to code "the research task that intervenes between 
d^a collection and analysis: " " ^ 



\, ^ ^Sheldon,, Brooke E.,ed.^^ Guide for tibrary Leaders, Staffs and Advisory Groups, produced under 

. a grant from the Bureau of Librarie^^d Learning Resources, U.S. Office of Education^: 1973 

I repr/>^uc'€d and distributed by The Continuing Library Education Network and Exchange, Box 1228 

' ' 620 Michigan Ave., NE, Washington, D.ffi. 20064. ' , . • - 
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interpretation: of "some current evaluatidn theory and translate it into a 
workable structure for practical application by training directbrs." p. 1. 

Stenner, A.Jackson, An Overvieto of Information Based Evaluation: A Design Procedure^ Arlingt( 
Virginia, Institute for Development of Educational Auditing, 1972, 37pp. ' ' ' ^ 

^ ■ • ■ ' ' /'.■•• • 

An advanced approach to the design of information based'^aluation. Jhe author^ 
. also contrast evaluation with research. A flow chart describes the devplopment of 
ail evaluatiori^esign. . • 



' EVALUATION OF EDUCATIONAL RESEARCH ^ 

Gephart, William J ., 2&d Bartos, Bruce B., Phi Delta Kappa Research Center Occasional Paper it 7 
Profiling Instructional Package, Phi Delta Kappa, August, 1969. 

Through the uSe of aResear&i Prtffilin'gnow Cha«t,\his evaluation ofeducatfonal 
research-will depitct sound procedures and weak*procedures. "Once the research . ^ 
user understands the strength of the pjocedures . /he ^^in a better position to use 
^ thd conclusions of that research in professional decisions. 

Ainswprth, Len, "Objective measure of the impact of a library learning center,^' School Libraries 
i8:33-aMAfetef^-1969. 

Maedgen Elementury^'School in LubbOck, Texas. From individual room' collections 
/ to centralized 'libniry - "To increase pupil proficiency in'the Use orflibrary." No 

'St^dardized test could be located - school- administre red a self-i/onstructed multiple . ^_ 
choice test. The control school adjacent elemenfary school in the same district. 
Reported significant improvement in the mean scqres of students jrpm pre-test to 
post-test* indicating an improvement in libr^^jkilT^pf participating students. No pre- . 
test at control scl<)ol. - , ' ; * 

^isholm, Robert L., "How to Ey^uate a' Good Library andJProgram,*' ScAoo/ Board Journal, 
24-25, 41, November, .]^65. .? 

Author points out objective measures and less than objective measures with 
emphasis on evaluation of pexSonnel. 



Graham^ Ma^, **Changingperspectivespn program evaluation," School Libraries, 18:27-2a. 
FalL 1968. ~ ' . ^ ' \ J' ^ 

, We shojajd not fear trvtJuation, Rather judgb the impact the program is makyig. / 
, Determining impact will provide infcftafnation necessary for decision making ^^jirij 
inj^erms of estal^Iished goals and objectives. The objectives of school nliedi^rob 

• ^ must be compatible with' objectives pf the school. Five pitfalls; 1) yjxo effecti\^|', 

- instrument for a completely objective evaluation of any segment of education. ^ 

* Ji) undertake toQ much at one time. Evaluate only on^ aspect at a time. 3) V 
hazstfd o.Fcomparison, even if^we have improved over yesterday, what impact Has 
dcgree^ of gr6wth made. 4r) We 4o not define. our goals. 5) Us- we ap^siye, deft 

^ * oversensitive, resistant to change. Evaluation if a creative,. on-going, cooperative 
I ' i leaming^pf ocess, essential for both Tong and short range planjiing a basis'for4>u€ 
^ requests - an action pf ogram^ecaHse tlje'results tell us directions to follow;, what 

' need to get them> changes which need to be rarade - areas to continued*emphasis. 
i I. foint in assessing impact uriless we are willing to take the action which is indicate 

Hale, Irlene W., '^Influence of library sei^i'c^ upon t'he academic achievement of twelftli gr^ 
st\Idknts,'' Wibon library kulletm^^ - * ^ • 

• . , . . ^ , • * 

Twelfth grade students --2 matched classes'-- oHe gol l^jrary resources in*^ varie 
the ot^er onl^ incidentally -I SO^siudents - group'ed by achieWment tests to gei 
ability representation. Both achievement and library skill test at'be^nning - M\ 
Research conducted through* Advanced Study fdr'Liljfary Personnel, .University t) 
Kentucky, Athens. ' MeasureJ^fifect iipbnthe learning of Students exp5se%to libta 



skill anH service.' 1^' X * > * 



Ik 



"How Well Are Yom Doiri^youi Job? Library Joilt^na^ SSi'Sbp^, December 15, 1958. ' 

J School library supervisor must continuously evaluate'the school libr^ progtam. 

Suggests measuring against national, regional aiicffbf^ate standards, conducting 
^ experimental anjd survey research. Als6 includes short questionnaire tq be self- 

administrerls^ concerning performance as a superviy)r. < * 

„ , atihg School Library Services,'' Zt^rary rrend5,;( X:372-385^ Jan 

, ' ' • I 4-' ^ 

Educational objectives of the school should be,<lire'cted^toward desired ^hangesT 
^ ^ * iathcb^ljIivioF of students: eyduition should provkte mejasqrement o.f the degre 
, „ effectiveness witb which an educational institution -* oPa componen^of it, ^.g. th 
^ ,y library - achieves such^objectives. Means designed for appraising school fibrary 
performance ar^ reviewecl, i.eu, ALA standards, state and regiohal accrediting 3£ssoi 
sidind^dsy £valuattve Criteria (Cooperative Study of Secondary School Standard|f) 
Henne's Planning Guick for a' High School Library Program considered. tfiV most t 
tool then available. Calls atterttion to the Illinois Coi^sensus Study^Proferam. / 

95 ^ ' r'/'^' 



Lewis, CoKvc D., "West Virginia Evaluates New School Libraries," West Virginia Libraries 
14:1-81, September, 1961. .... 

Reviews results of survey bf West Vir^a school libraries, in 1960 which^led tq 
West Virginia's selectioii as one of the states to receive aid in implementing the 
I960 Standards for School Library Programs (ALA) 

^ Lpertschcr, David V. and Land, PhyUis, "An Empirical Study of Medif Services in Indi3na ' 
^^^^prii^Sdiools," School Media Quarterly/ 4:8-18, FsJl 197 5 ' ' 



A report of a study measuring perceptions of students, teachers and media staff 
concerning services received>oin the elementary media center. * , . 

Lowrey, Anna Mary, "Staffing Patterns and Education for Media Center Personnel: Relevant 
or Regressive?" Iii'rary Tremir, 19's509-519, April, 1971. ' 

■ Calls for a reform iii library education and a redefinition of the library, function in the 
• light of increasing use of media. Task analysi's is required to determine Who is doirfg 
what, who should be doing wh^t and should it be done at all. Recbmmends applying 
the systems approach to curriculum deveioprnent in library schools. Curriculum <- 
needs to be revised so that graduates have competency m printed and audiovisual 
forms 6f communication and their accompanying technologies, and competency " 
in human behavior, learning theory, management, planning, evaluation, and research. 
Jf • Programs for staff developplent are also essential. 



